Programmatically create RabbitMQ Exchange and Queue with Burrow.NET

- As mentioned in previous posts, IRouteFinder is an interface defined in Burrow.NET to help publishing the message to correct target. If you have many message types, potentially you will need many exchanges and queues. And if it's true, setup everything for different environments or different machines is not an interesting task. So with Burrow.NET we can programmatically create these exchanges, queues and bind them very easy.

- What you need is an implementation of IRouteFinder which should define your exchange names, queues name based on message type.

class TestRouteFinder : IRouteFinder
{
    public string FindExchangeName<T>()
    {
        return "Burrow.Exchange";
    }

    public string FindRoutingKey<T>()
    {
        return typeof(T).Name;
    }

    public string FindQueueName<T>(string subscriptionName)
    {
        return string.IsNullOrEmpty(subscriptionName)
            ? string.Format("Burrow.Queue.{0}", typeof(T).Name)
            : string.Format("Burrow.Queue.{0}.{1}", subscriptionName, typeof(T).Name);
    }
}

- Once having the route finder implemented, write a few lines of code like below:

var exchangeSetupData = new ExchangeSetupData{ExchangeType = "direct"};   
var queueSetupData = new QueueSetupData {SubscriptionName = "BurrowTestApp", MessageTimeToLive = 100000};

Func<string, string, IRouteFinder> factory = (environment, exchangeType) => new TestRouteFinder();
var setup = new RabbitSetup(factory, Global.DefaultWatcher, ConfigurationManager.ConnectionStrings["RabbitMQ"].ToString(), "TEST");

setup.SetupExchangeAndQueueFor<YourMessageA>(exchangeSetupData, queueSetupData);
setup.SetupExchangeAndQueueFor<YourMessageB>(exchangeSetupData, queueSetupData);
setup.SetupExchangeAndQueueFor<YourMessageC>(exchangeSetupData, queueSetupData);

- Ofcourse, if you have a predefined exchange and queue names but want to have the same settings for different box you can use built-in ConstantRouteFinder:

Func<string, string, IRouteFinder> factory = (environment, exchangeType) => new ConstantRouteFinder("exchangeName", "queueName", "routingKey");

- These code can be run multi times even though the exchanges and queues have've been created before. In practice, I use a application config key/value to determine to run these setup or not. If there have been existing exchanges, queues with the same names but the setting details like ExchangeType, MessageTimeToLive, Durable are different, running the code against the same server will print out the warning messages that would tell you the settings are different. Please note that different exchange type will lead to unexpected results.

- For someone who like to write integration tests which need to create the exchanges, queues before a test and destroy everything after the test, you can use the Destroy method:

Func<string, string, IRouteFinder> factory = (x, y) => new TestRouteFinder();
var setup = new RabbitSetup(factory, Global.DefaultWatcher, ConfigurationManager.ConnectionStrings["RabbitMQ"].ToString(), "TEST");

setup.Destroy<YourMessageA>(exchangeSetupData, queueSetupData);
setup.Destroy<YourMessageB>(exchangeSetupData, queueSetupData);
setup.Destroy<YourMessageC>(exchangeSetupData, queueSetupData);

- This sample code is also available in Githup project


Cheers

Read more »

How I Mock Mongo Collection?

- I have been working with Mongo quite alot recently and writing my service classes using mongo without testing is such a pain in the neck. So today, I'll post about how I mock mongo collection for my unit testings.
- Given that I have a ReminderService like below:

public class ReminderService
{
    private readonly MongoCollection<User> _mongoCollection;
    private readonly INotificationService _notificationService;

    public ReminderService(MongoCollection<User> mongoCollection, INotificationService notificationService)
    {
        _mongoCollection = mongoCollection;
        _notificationService = notificationService;
    }

    public void Send()
    {
        var notActivatedUsers = _mongoCollection.FindAs<User>(Query.EQ("activated", false)).ToList();
        notActivatedUsers.ForEach(user => _notificationService.Send(new ActivationReminder(user)));
    }
}

- It has 1 method Send which needs to be unit tested. This method will find all users who have not their activated account and then send reminder emails to them. This service class is oviously designed for unit testing as it has 2 dependencies: a mongo collection and an interface of INotificationService. To mock the interface, it's straightforward , isn't it? So our job today is mocking this mongo collection.

- Before mocking a class like mongo collection, I often check whether it has virtual methods because the thing we need to mock here is the method FindAs. By pressing F12 on the MongoCollection class to go to assembly metadata, we can easily see that this method is virtual and return a MongoCursor.

// Summary:
//     Returns a cursor that can be used to find all documents in this collection
//     that match the query as TDocuments.
//
// Parameters:
//   query:
//     The query (usually a QueryDocument or constructed using the Query builder).
//
// Type parameters:
//   TDocument:
//     The type to deserialize the documents as.
//
// Returns:
//     A MongoDB.Driver.MongoCursor<TDocument>.
public virtual MongoCursor<TDocument> FindAs<TDocument>(IMongoQuery query);

- Hmm, it's getting more complicated here because this method does not return an enumerable or collection but a MongoCursor<TDocument> and that type inherits from IEnumerable<TDocument>. So it's telling me that when we call mongoCursor.ToList(), behind the scene, it will call GetEnumerator(). And certainly, checking class MongoCursor, I see that it's methods are all virtual so it's really open for unit testing.

- To wrap up, the idea will be creating a mock cursor to return expected items and let it be the result of method FindAs on our mock collection. There are other things to note here: mocking a class is not similar to mocking an interface since it has constructor. That means we have to provide the constructor parameters to create a mock object of MongoCollection class. The parameters here are MongoDatabase and MongoCollectionSettings<TDefaultDocument>:

// Summary:
//     Creates a new instance of MongoCollection. Normally you would call one of
//     the indexers or GetCollection methods of MongoDatabase instead.
//
// Parameters:
//   database:
//     The database that contains this collection.
//
//   settings:
//     The settings to use to access this collection.
public MongoCollection(MongoDatabase database, MongoCollectionSettings<TDefaultDocument> settings);

- To have a MongoDatabase we need MongoServer and MongoDatabaseSettings and so on. So honestly it's not really simple. I don't want my unit test to talk to real mongo server while running so I must ensure everything runs in memory. I had to create a TestHelper class for these thing, ofcourse for reuse reason:

public static class MongoTestHelpers
{
    private static readonly MongoServerSettings ServerSettings;
    private static readonly MongoServer Server;
    private static readonly MongoDatabaseSettings DatabaseSettings;
    private static readonly MongoDatabase Database;

    static MongoTestHelpers()
    {
        ServerSettings = new MongoServerSettings
        {
            Servers = new List<MongoServerAddress>
            {
                new MongoServerAddress("unittest")
            }
        };
        Server = new MongoServer(ServerSettings);
        DatabaseSettings = new MongoDatabaseSettings("databaseName", new MongoCredentials("", ""), GuidRepresentation.Standard, SafeMode.True, true);
        Database = new MongoDatabase(Server, DatabaseSettings);
    }

    /// <summary>
    /// Creates a mock of mongo collection
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <returns></returns>
    /// <remarks></remarks>
    public static MongoCollection<T> CreateMockCollection<T>()
    {
        var collectionSetting = new MongoCollectionSettings<T>(Database, typeof(T).Name);
        var m = Substitute.For<MongoCollection<T>>(Database, collectionSetting);
        m.Database.Returns(Database);
        m.Settings.Returns(collectionSetting);
        return m;
    }

    public static MongoCollection<T> ReturnsCollection<T>(this MongoCollection<T> collection, IEnumerable<T> enumerable)
    {
        var cursor = Substitute.For<MongoCursor<T>>(collection, Substitute.For<IMongoQuery>());
        cursor.GetEnumerator().Returns(enumerable.GetEnumerator());
        cursor.When(x => x.GetEnumerator())
              .Do(callInfo => enumerable.GetEnumerator().Reset());// Reset Enumerator, incase the method is called multiple times

        cursor.SetSortOrder(Arg.Any<IMongoSortBy>()).Returns(cursor);
        cursor.SetLimit(Arg.Any<int>()).Returns(cursor);
        cursor.SetFields(Arg.Any<IMongoFields>()).Returns(cursor);
        cursor.SetFields(Arg.Any<string[]>()).Returns(cursor);
        cursor.SetFields(Arg.Any<string>()).Returns(cursor);
        cursor.SetSkip(Arg.Any<int>()).Returns(cursor);

        collection.Find(Arg.Any<IMongoQuery>()).Returns(cursor);
        collection.FindAs<T>(Arg.Any<IMongoQuery>()).Returns(cursor);
        collection.FindAll().Returns(cursor);
        // You properly need to setup more methods of cursor here
        
        return collection;
    }
}

- There are some interesting points here:

1. I'm using NSubstitute, hope you understand the syntax. I think the syntax is similar to other frameworks like Moq so it's not a problem if you use different library.
2. The code will never talk to real database as I used "unittest" as server address.
3. Method ReturnsCollection is an extension method of MongoCollection<T>. We'll use it to specify the items we simulate the result from mongo query. In practice, we might use fluent syntax on mongo cursor object such as SetSortOrder, SetLimit, etc so we definitely don't want these things alter our result. That's why you see there're alot of lines to mock the result for those methods to return the same cursor.

- So with this helper, the unit test will look like this:

[TestFixture]
public class ReminderServiceTests
{
    [Test]
    public void Send_should_send_notification_to_unactivated_users()
    {
        // Arrange
        var mockCollection = MongoTestHelpers.CreateMockCollection<User>();
        mockCollection.ReturnsCollection(new List<User>
        {
            new User {IsActivated = false}, 
            new User {IsActivated = false}
        });
        var notificationService = Substitute.For<INotificationService>();
        var service = new ReminderService(mockCollection, notificationService);

        // Action
        service.Send();

        // Assert
        notificationService.Received(2).Send(Arg.Any<ActivationReminder>());
    }
}

- Well, The purpose of this test is guaranteeing that the notification service will send reminders to all unactivated users from database. Because I'm using mongo so the final thing we want to mock is the result set returned by Mongo, we're not testing mongo query here. Hope this help.


Cheers.

Read more »

Save a little time when writing unit test using VS.NET Macro

- As a unit testing passioner, I write a lot of unit tests for my code. But repeating adding test attributes to classes and methods, adding resharper ignore InconsistentNaming statement or adding AAA pattern are killing me. So I thought of making some awesome macros to save my time.
- Basically, when I add a test class, the file will originaly be like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace MyProject
{
    class NewTestClass
    {
    }
}

- And I want it to be like this after a few key presses:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.VisualStudio.TestTools.UnitTesting;

// ReSharper disable InconsistentNaming
namespace MyProject
{
    [TestClass]
    public class NewTestClass
    {
        [TestMethod]
        public void Should_be_implemented_NOW()
        {
            // Arrange


            // Action


            // Assert

        }
    }
}
// ReSharper restore InconsistentNaming

- This is archivable with following macros:

Sub CreateTestClassAttribute()
 DTE.ActiveDocument.Selection.StartOfLine(vsStartOfLineOptions.vsStartOfLineOptionsFirstText)
    DTE.ActiveDocument.Selection.Text = "public "
    DTE.ActiveDocument.Selection.StartOfLine(vsStartOfLineOptions.vsStartOfLineOptionsFirstText)
    DTE.ActiveDocument.Selection.NewLine()
    DTE.ActiveDocument.Selection.LineUp()
    DTE.ActiveDocument.Selection.Text = "[TestClass]"
    DTE.ActiveDocument.Selection.EndOfLine()
    DTE.ActiveDocument.Selection.LineUp(False, 3)
    DTE.ActiveDocument.Selection.StartOfLine(vsStartOfLineOptions.vsStartOfLineOptionsFirstText, True)
    DTE.ActiveDocument.Selection.Text = "using Microsoft.VisualStudio.TestTools.UnitTesting;"
    DTE.ActiveDocument.Selection.EndOfLine()
    DTE.ActiveDocument.Selection.NewLine()

    DTE.ActiveDocument.Selection.NewLine()
    DTE.ActiveDocument.Selection.Text = "// ReSharper disable InconsistentNaming"
    DTE.ActiveDocument.Selection.EndOfLine()
    DTE.ActiveDocument.Selection.LineDown(False, 8)
    DTE.ActiveDocument.Selection.EndOfLine()
    DTE.ActiveDocument.Selection.Text = "// ReSharper restore InconsistentNaming"
    DTE.ActiveDocument.Selection.LineUp(False, 3)

    CreateTestMethod()
End Sub
Public Sub CreateTestMethod()
    DTE.ActiveDocument.Selection.NewLine()
    DTE.ActiveDocument.Selection.Text = "[TestMethod]"
    DTE.ActiveDocument.Selection.NewLine()
    DTE.ActiveDocument.Selection.Text = "public void Should_be_implemented_now()"
    DTE.ActiveDocument.Selection.NewLine()
    DTE.ActiveDocument.Selection.Text = "{"
    DTE.ActiveDocument.Selection.NewLine()

    DTE.ActiveDocument.Selection.Text = "// Arrange"
    DTE.ActiveDocument.Selection.NewLine(3)

    DTE.ActiveDocument.Selection.Text = "// Action"
    DTE.ActiveDocument.Selection.NewLine(3)

    DTE.ActiveDocument.Selection.Text = "// Assert"
    DTE.ActiveDocument.Selection.NewLine()
    DTE.ActiveDocument.Selection.LineUp(False, 6)
End Sub

- Well, the steps are simple:

1. In your VS.NET click Tool -> Macros -> Macro Explorer and paste these macros to the RecordingModule.
2. Choose Tool again -> Options -> Keyboard. Type in Macros, choose these 2 macros and bind your favorite hot key for it like picture below:

- I choose Ctrl F - Ctrl C to make a test class with first test method and Ctrl F - Ctrl M to add another Test method.
- Hehe, It's DRY principle in unit testing. I has been using different Unit testing frameworks like NUnit, MBUnit, etc but in this post, I used VS.NET test framework. You properly need to alter the macros a little bit to fit your needs. Now enjoy your tests. Cheers

Read more »

Things you can easily change in Burrow.NET

- As I said in other posts, Burrow.NET aims to generic needs of any .NET projects that use RabbitMQ. I believe the built-in code has covered most of your requirements for common usages. However there are some things you can easily change without writing any new code or at least just a few lines of code. So in this post, I'll introduce about things you can make to suit your needs when working with RabbitMQ using Burrow.NET.

1/ Global.DefaultConsumerBatchSize

- The default value of this static property is 256. When you use ITunnel object to asynchronously subscribe to a queue, Burrow.NET use a internal Semaphore object when creating asynchronous Task to handle the messages from the queue. The amount of Tasks it create will depend on this value since it's used as the initialize value for the internal Semaphore. In the mean time, this value is also used to call IModel.BasicQos, which eventually sets the maximum amount of messages stay on the Unacknowledged list when they are consumed. That's the fetchsize value mentioned in RabbitMQ document and they advise to keep this number as high as double to the acknowledge speed. Therefore, you potentially should adjust this number to fit your need. Moreover, the amount of Tasks and eventualy Threads Burrow.NET create when consumming messages is in proportion to this value and I personally think creating more threads does not make the application run faster. I'll keep monitoring the way it works in my projects so I could split the value for the queue fetch size and the number of concurrent Tasks if it's neccessary but I think so far so good.

2/ Global.DefaultPersistentMode

- This value is equal to true by default and is used when creating the ITunnel object. It's eventually used when you publish messages. If the persistent mode is true, the message will be written to disk instead of being kept in memory. So if there is any problem with the server and you have the machine reboot, your messages will be safe. Otherwise, they will be removed when server reboots. However, keeping messages in memory could improve the performance. The DefaultPersistentMode is used to create ITunnel object in TunnelFactory, but you can change it anytime as the ITunnel has a method to do that.

3/ Global.DefaultErrorExchangeName and Global.DefaultErrorQueueName

- The built-in error handler will publish any unhandled exception wrapped in an object and publish to these exchange/queue so you can modify this value to route the error messages to their new home. The default value for the exchange is "Burrow.Exchange.Error" and "Burrow.Queue.Error" for the queue.

4/ Global.DefaultSerializer

- This property hold the default value of an JavaScriptSerializer which is quite slow in performance and cannot handle serialization very well. The reason to have this is to reduce the dependencies to 3rd libraries so you can just use Burrow.dll without Burrow.Extras.dll. However, I know everyone likes Json.NET include me so there is another implementation of ISerializer locates in Burrow.Extras package using Json.NET. It's not the fastest .NET serializer but fast enough for common usages. So I strongly recommend using this one in your project. Again, this value is used to create ITunnel and the ITunnel object will have a method to change that anytime.

Global.DefaultSerializer = new JsonSerializer();

var tunnel = RabbitTunnel.Factory.Create();
tunnel.SetSerializer(new JsonSerializer());

5/ Global.DefaultWatcher

- This one is used to write debug messages from Burrow.NET. The default value is ConsoleWatcher which writes messages to console so if you're writing a console app, you will easily see those messages. You properly want to change it to anything else such as Log4Net. To do that, simply implement IRabbitWatcher using Log4net and change the DefaultWatcher when app start.

Global.DefaultWatcher = new Log4NetRabbitWatcher();

6/ Global.DefaultCorrelationIdGenerator

- This one is used to generate the CorrelationId when you publish a message. The default implementation will generate a Guid and I think it's pretty enough for an Id. You don't have to change that but just do it if you have time and good reason

7/ Global.DefaultTypeNameSerializer

- Like the CorrelationId, this one is used to generate a string to determine the message when you publish messsags to RabbitMQ. I can think of a case we might need this value is when you just publish any type of message to the queue. When consuming the messages, base on this value you can know the type and deserialie the object. But honestly I've never tried this, neither used this value so if you don't need it in your project, leave it.

- In conclusion, I put these things in the Global class just to make it's easier to change the behavior of the ITunnel without touching to the TunnelFactory or calling methods on ITunnel object. Things like DefaultWatcher, DefaultTypeNameSerializer and DefaultCorrelationIdGenerator are the bare minimum needs for Burrow.NET and you can simply use the library without worrying about those values. Again, the only thing I suggest to take care of is the DefaultSerializer, just change it to JsonSerializer for better performance.

Cheers.

Read more »

Custom Burrow.NET TunnelFactory & RabbitMQ

I/ Default TunnelFactory

- When using Burrow.NET, one of the important thing to create is a Tunnel by using TunnelFactory:

var tunnel = RabbitTunnel.Factory.Create();

- The Factory static property of RabbitTunnel keeps a default instance of TunnelFactory class. The TunnelFactory has 4 overloads to create the ITunnel object:

public virtual ITunnel Create();
public virtual ITunnel Create(string connectionString);
public virtual ITunnel Create(string connectionString, IRabbitWatcher watcher):
public virtual ITunnel Create(string hostName, string virtualHost, string username, string password, IRabbitWatcher watcher):

- They're all virtual so if you want to implement a custom ITunnel, you have things to override. Personally, I don't think of any cases to make a custom ITunnel but just in case you want to change some thing . These 4 methods are quite straight forward and I think the first one is the simplest one to use since it'll take the RabbitMQ connection string from app.config and create an built-in ITunnel object with built-in dependencies.

- I intentionally don't create an ITunnelFactory but only this base class. The reason is I tend to keep only 1 type of TunnelFactory in use in the whole AppDomain. I mean if you create a CustomTunnelFactory which inherits from class TunnelFactory, you just need to initialize an instance of your class somewhere and this action will automatcally set RabbitTunnel.Factory to this new factory. That's the only way to change the type of RabbitTunnel.Factory since it does not have public setter. Well you still can create any static classes that have similar methods to create the ITunnel if you want but in my opinion, creating tunnel objects from RabbitTunnel.Factory looks better.

II/ Implement a custom TunnelFactory and make it as default

- Let's say you want to replace default DefaultRouteFinder of Burrow.NET and set it your one as the route finder whenever you create an ITunnel. There are at least 2 ways to do that. The first way is creating a custom Factory and override the last method of 4 methods above since that built-in method always use DefaultRouteFinder as RouteFinder. The second way is using DependencyInjectionTunnelFactory. I'll talk about the 2nd way later in this post. There is another simple method to change the RouteFinder of an ITunnel, that is using the method SetRouteFinder of ITunnel but in this topic, we're talking about make it as default RouteFinder.

- This is how to implement the first method:

public class CustomTunnelFactory : TunnelFactory
{
    public override ITunnel Create(string hostName, string virtualHost, string username, string password, IRabbitWatcher watcher)
    {
        var rabbitWatcher = watcher ?? Global.DefaultWatcher;
            var connectionFactory = new RabbitMQ.Client.ConnectionFactory
                                        {
                                            HostName = hostName,
                                            VirtualHost = virtualHost,
                                            UserName = username,
                                            Password = password
                                        };

            var durableConnection = new DurableConnection(new DefaultRetryPolicy(), rabbitWatcher, connectionFactory);
            var errorHandler = new ConsumerErrorHandler(connectionFactory, Global.DefaultSerializer, rabbitWatcher);
            var msgHandlerFactory = new DefaultMessageHandlerFactory(errorHandler, rabbitWatcher);
            var consumerManager = new ConsumerManager(rabbitWatcher, msgHandlerFactory, Global.DefaultSerializer, Global.DefaultConsumerBatchSize);

            return new RabbitTunnel(consumerManager,
                                    rabbitWatcher, 
                                    new CustomRouteFinder(), 
                                    durableConnection,
                                    Global.DefaultSerializer,
                                    Global.DefaultCorrelationIdGenerator,
                                    Global.DefaultPersistentMode);
    }
}

- And here is how to register this as default and use it:

// Put this line somewhere when app start:
new CustomTunnelFactory();

// Create your tunnel:
RabbitTunnel.Factory.Create();

// Hmmm, I can't stop you doing this:
(new CustomTunnelFactory()).Create(); // But please, it's such an ugly code

- This first way could possibly require you to read the the original TunnelFactory source code to know how to replace the route finder. In fact the override method is exactly the same as the base one except the CustomRouteFinder thing. I personally don't like this way since duplication is always bad. That's why we have DependencyInjectionTunnelFactory.

III/ Dependency Injection Tunnel Factory

- Dependecy Injection is my favorite topic. I like to use libraries that allow me to change behavior easily and I tend to do the same thing in my code. The default RabbitTunnel class has some dependencies. As it's constructor requires following things:

• IConsumerManager
• IRabbitWatcher
• IRouteFinder
• IDurableConnection
• ISerializer
• ICorrelationIdGenerator

- Again, these interfaces have default implementations and I cannot think of any cases that have to change the behavior, but I would not close the door to change it. The potential behavior you possibly want to change could be the way this library handle error which is implemented in ConsumerErrorHandler class. There is a long chunk of dependencies to get there so properly using DependencyInjectionTunnelFactory is the easiest way to change the behavior without spending time reading the whole source code. The way DependencyInjectionTunnelFactory works is that it will try to resolve the object it requires. If it cannot find any registed dependencies, it'll use the default one.

- To use this factory, you have to implement IBurrowResolver interface. Basically, it exposes methods like some popular IOC library like Autofac or StructureMap. So if you use Autofac in your project, simply imlement IBurrowResolver and wrap the Autofac's IContainer inside, delegate all methods of IBurrowResolver to IContainer.

- After have the IBurrowResolver implemented, here is how to use it:

RabbitTunnel.Factory.RegisterResolver(new AutofacBurrowResolver());
var tunnel = RabbitTunnel.Factory.Create();

- That's it. Behind the scene, this extension method does exactly what I say above:

namespace Burrow.Extras
{
    public static class TunnelFactoryExtensions
    {
        /// <summary>
        /// Call this method to register a dependency resolver and set default TunnelFactory to DependencyInjectionTunnelFactory
        /// </summary>
        /// <param name="factory"></param>
        /// <param name="burrowResolver"></param>
        public static void RegisterResolver(this TunnelFactory factory, IBurrowResolver burrowResolver)
        {
            new DependencyInjectionTunnelFactory(burrowResolver);           
        }
    }
}

- These things are located in Burrow.Extras so you propably have to grab that nuget package to use it. Obviously, you have to register any dependencies before creating any ITunnel. For instance, if you're using Autofac and you want to change the type of RabbitWatcher when creating ITunnel you have to do this first:

var builder = new ContainerBuilder();
builder.RegisterType<CustomRabbitWatcher>()
       .As<IRabbitWatcher>()
       .SingleInstance();

- And it's pretty similar for other IOC libraries. I use the above RabbitWatcher replacement thing as an example because there is definitely a simpler way to replace the default watcher type which is setting it to Global static class. Okey, It's enough for this topic. I would love to hear any suggestions & comments from you to make this better.

Cheers.

Read more »