search

SzymonPobiega / NServiceBus.Router

Cross-transport, cross-site and possibly cross-cloud router component for NServiceBus
MIT License
5 stars 10 forks source link

Add support to route to multiple ASB topics in the same namespace #48

Open marcselis opened 2 years ago

marcselis commented 2 years ago

When setting up a router between a multi-schema SQL transport & a multi-topic ASB transport all subscriptions are created in the routers own ASB topic, instead of the topic where the event is published, causing the SQL subscribers not to receive the event. Please see Azure Service Bus Transport issue #511 for more context.

SzymonPobiega commented 2 years ago

Hi @marcselis

I replied in the other thread in the ASB transport repo. If I understand correctly your scenario, it is possible to configure it with the current version of the Router.

marcselis commented 2 years ago

Ok, I'm currently sick at home, but I'll have a look at it when I'm back at work. Thanks!

marcselis commented 2 years ago

If I understand your sample correctly, you propose to setup a router with 1 channel for SQL transport and a separate channel for each ASB topic. This can work, but we currently have 22 schema's in the SQL transport and I expect in a few years to also reach that same number of topics.
Do you think this setup is scalable enough? A single router with 23 channels? And if the ASB endpoints want to send messages to SQL endpoints residing in different schema's, will the single SQL channel be suffucient? Or will they also need to be split-up in different channels?

SzymonPobiega commented 2 years ago

Do you think this setup is scalable enough? A single router with 23 channels?

Yes, I don't see any obvious problems there. An alternative solution could be to try to create a forwarding role between topics. I think that should be possible to create in Azure Service Bus. This is not supported by Router or NServiceBus so it would require manual maintenance but given you are going to make a heavy use of it, it might be worth developing a custom little tool that can connect the topics with forwarding and manage that.

And if the ASB endpoints want to send messages to SQL endpoints residing in different schema's, will the single SQL channel be suffucient?

One interface on the SQL side is enough because the SQL Server transport is catalog-aware. It can router between catalogs and schemas without external help.

marcselis commented 2 years ago

Thanks @SzymonPobiega for the confirmation, but I think this is way to overcomplicated.

Topics in ASB are not used to physically divide endpoints in groups. They are only used while publishing an event. The subscriptions for a topic and their filters determine in what queue the published event will be placed for processing.

When routing a message or command between SQL and ASB, the topic is not important as the message can be deliverd directly in the destination endpoint's queue. The same is true when routing a message between ASB & SQL. So, the whole message routing can be done using a single ASB channel, which is a much simpler setup.

The 'topic' problem only comes into play when a SQL endpoint wants to subscribe to an event published by an ASB endpoint. The ASB channel needs to know in what topic the subscribed event is published, so it can create the correct subscription. The current router setup uses a "destination endpoint" to determine what channel to use to forward the subscription. But most modern transports like ASB, RabbitMQ and even SQL Server don't need to know the exact endpoint that publishes the event because the transport itself determines where to send a certain event to, based on centrally stored subscription information. In the case of ASB & RabbitMQ, these subscriptions are stored in the Topic. In case of SQL server, they are stored in a central subscriptionrouting table. In your samples the topic to send a subscription to can easily be determined based on the destination endpoint's name: endpoint's name starts with 'Green.' ==> topic is 'Green'. In our case there is no relation whatsoever between an endpoint's name and the topic it is using to publish messages, so there is no way to determine the correct ASB channel except creating a mapping of all ASB endpoint queues and the topic they are using.

It would be much simpler if the "destination part" of the subscription message would just contain the destination that will handle the subscription. In an ASB or RabbitMQ channel this could just be as simple as the Topic name where to create the subscription in. That would eliminate the need for having all those different ASB channels.