RabbitMQ to Azure Service Bus events don't get routed

[bwhalversen]

I've created a simple sample solution with three console applications: a RabbitMQ endpoint, a RabbitMQ to Azure Service Bus (ASB) router, and an ASB endpoint. The router successfully routes an NServiceBus ICommand from the RabbitMQ endpoint to the ASB endpoint. Works great with no issues! However, if the RabbitMQ endpoint attempts to publish an IEvent that is subscribed to by ASB endpoint, the event never makes it's way to ASB. I assume this is meant to work? Both RabbitMQ and ASB have native pub/sub ability.

I can see that most of the necessary infrastructure for publishing the IEvent is set up properly:

1. In RabbitMQ an exchange is created for the IEvent to be published, like expected.
2. I also see in RabbitMQ exchanges and queues created for both the RabbitMQ endpoint and the router endpoint.
3. I see in the Azure portal queues created for both the ASB endpoint and the router, as expected.
4. I see in the Azure portal my topic created with a proper subscription for my IEvent.

However, the published event never makes it's way to my ASB endpoint. Upon further investigation, it appears the problem may be with the RabbitMQ side of the communication. I found that if I manually create a binding between the exchange for my IEvent and the router exchange (which was automatically created) then my published IEvent makes it's way to the ASB endpoint as expected. This seems to indicate that the issue may lie in a binding not being properly created in RabbitMQ for forwarding IEvents to the router exchange.

This behavior can be easily confirmed. I have simple code that can be made available that demonstrates this. So my questions are these:

1. Are IEvents published by a RabbitMQ endpoint meant to work with NServiceBus.Router?
2. If so, what can I do so that the IEvents I publish from my RabbitMQ endpoint will be bound to the router endpoint and be forwarded? Is there some NServiceBus.Router option/configuration/trick I can use so that my published events will be forwarded to the router and make their way to ASB? I assume I must be overlooking something obvious but I can't see what it is.

Here is the core code of my sample:

RabbitMQ endpoint configuration:

            var endpointConfiguration = new EndpointConfiguration(Settings.SampleClientEndpointName);

            endpointConfiguration.License(NServiceBusLicense.Text);
            endpointConfiguration.UseSerialization<NewtonsoftSerializer>();
            endpointConfiguration.UsePersistence<InMemoryPersistence>();
            endpointConfiguration.EnableInstallers();
            var recoverability = endpointConfiguration.Recoverability();
            recoverability.Immediate(immediate => immediate.NumberOfRetries(0));
            recoverability.Delayed(delayed => delayed.NumberOfRetries(0));
            endpointConfiguration.SetDiagnosticsPath($@"C:\Temp\NSB\{Settings.SampleClientEndpointName}");
            endpointConfiguration.EnableInstallers();

            var routingSettings = endpointConfiguration.UseTransport<RabbitMQTransport>()
                .ConnectionString(Settings.RabbitMqConnectionString)
                .UseConventionalRoutingTopology()
                .Transactions(TransportTransactionMode.ReceiveOnly)
                .Routing();

            var bridge = routingSettings.ConnectToRouter(Settings.SampleRouterEndpointName);
            bridge.RouteToEndpoint(typeof(TestCommand), Settings.SampleServerEndpointName);

            var endpointInstance = await Endpoint.Start(endpointConfiguration).ConfigureAwait(false);

RabbitMQ to ASB router configuration:

            var routerConfig = new RouterConfiguration(Settings.SampleRouterEndpointName);

            routerConfig.AddInterface<RabbitMQTransport>(RabbitMqInterfaceName, t =>
            {
                t.ConnectionString(Settings.RabbitMqConnectionString);
                t.UseConventionalRoutingTopology();
            });

            routerConfig.AddInterface<AzureServiceBusTransport>(AzureServiceBusInterfaceName, t =>
            {
                t.ConnectionString(Settings.AzureServiceBusConnectionString);
                t.TopicName(Settings.TopicName);
            });

            var staticRouting = routerConfig.UseStaticRoutingProtocol();
            staticRouting.AddForwardRoute(RabbitMqInterfaceName, AzureServiceBusInterfaceName);

            routerConfig.AutoCreateQueues();

            var router = NServiceBus.Router.Router.Create(routerConfig);

            await router.Start().ConfigureAwait(false);

Azure Service Bus endpoint configuration:

            var endpointConfiguration = new EndpointConfiguration(Settings.SampleServerEndpointName);

            endpointConfiguration.License(NServiceBusLicense.Text);
            endpointConfiguration.UseSerialization<NewtonsoftSerializer>();
            endpointConfiguration.UsePersistence<InMemoryPersistence>();
            endpointConfiguration.EnableInstallers();
            var recoverability = endpointConfiguration.Recoverability();
            recoverability.Immediate(immediate => immediate.NumberOfRetries(0));
            recoverability.Delayed(delayed => delayed.NumberOfRetries(0));
            endpointConfiguration.SetDiagnosticsPath($@"C:\Temp\NSB\{Settings.SampleServerEndpointName}");

            endpointConfiguration.UseTransport<AzureServiceBusTransport>()
                .ConnectionString(Settings.AzureServiceBusConnectionString)
                .Transactions(TransportTransactionMode.ReceiveOnly)
                .EnablePartitioning()
                .PrefetchMultiplier(10)
                .TopicName(Settings.TopicName);

            var endpointInstance = await Endpoint.Start(endpointConfiguration).ConfigureAwait(false);

And again, let me point out, that if I manually create a binding in RabbitMQ between the IEvent exchange and the router exchange then my events DO work. I just shouldn't need to manually create this binding for events to work. I need to understand how the router, or my use of it, will automatically create the necessary exchange bindings for IEvents in RabbitMQ.

Any insight that you can provide will be greatly appreciated.

[SzymonPobiega]

Hi

Which version of the Router and Router.Connector do you use? In the latest (3.10), we added automatic subscriptions. In order for them to work you need to explicitly connect the ASB endpoint to the Router by calling the code similar to the one in the RabbitMQ endpoint:

var bridge = routingSettings.ConnectToRouter(Settings.SampleRouterEndpointName);

The general rule of thumb is, you need to call ConnectToRouter in all the endpoints that send commands over router and ones that subscribe to events. You don't need to call that API in endpoints that only publish events or receive commands but I would advice to call it consistently in all your endpoints because you never no when another team member adds another handler etc.

In previous versions, in addition to the connect API above you had to also explicitly specify the publisher of the event like this:

var bridge = transport.Routing().ConnectToRouter(Settings.SampleRouterEndpointName);
bridge.RegisterPublisher(typeof(MyEvent), "ThePublisher");

but it is not needed any more, unless the publisher side transport does not have native Pub/Sub support (e.g. MSMQ).

[bwhalversen]

Thanks for the pointer. That did the trick. My events are now getting routed properly. It would be good to work into the documentation this paragraph somewhere since it is succinct and very illuminating:

"The general rule of thumb is, you need to call ConnectToRouter in all the endpoints that send commands over router and ones that subscribe to events. You don't need to call that API in endpoints that only publish events or receive commands but I would advice to call it consistently in all your endpoints because you never no when another team member adds another handler etc."

There was nothing in the documentation that suggested to my mind that a subscriber endpoint, that sends no commands, would still need to connect to the router because it has no commands to route. Maybe I missed this but making it explicit like in the paragraph above would be very helpful.

Thanks for your prompt help!

[bwhalversen]

A follow-up question based on the POC I'm working on. Is it possible to somehow configure a router to "listen" for all messages placed in a queue for one transport/interface (i.e. RabbitMQ) and automatically forward them to another transport/interface (i.e. ASB)? That is, a routing that would simply forward ALL messages from a queue in one transport to a queue in a different transport.

[SzymonPobiega]

Sorry for the late answer. I somehow missed the notification. That scenario is not supported out-of-the box by the router but I think in principle you can configure the router to do so. Why would that be a simplification? What is the scenario you are trying to address?

[bwhalversen]

Thanks for the reply. As our requirements got better defined I no longer need to attempt this so the question can be ignored. Thanks for reaching out nevertheless. BTW the router has been working quite well for us thus far. Very useful.