awslabs / aws-dotnet-messaging

An AWS-native framework that simplifies the development of .NET message processing applications that use AWS services, such as SQS, SNS, and EventBridge.
https://awslabs.github.io/aws-dotnet-messaging/
Apache License 2.0
81 stars 15 forks source link

Send messages or publish events to multiple destinations? #123

Open ashovlin opened 8 months ago

ashovlin commented 8 months ago

Describe the feature

Currently for configuration like the following:

services.AddAWSMessageBus(builder =>
{
    bus.AddSQSPublisher<ChatMessage>("https://sqs.us-east-1.amazonaws.com/012345678910/QueueA");
    bus.AddSQSPublisher<ChatMessage>("https://sqs.us-east-1.amazonaws.com/012345678910/QueueB" );
});

When sending a ChatMessage, the framework will send it to QueueA because it happened to be registered first.

https://github.com/awslabs/aws-dotnet-messaging/blob/db7703ff5eb4a17c7450ee55713b41c7cac2fc4a/src/AWS.Messaging/Configuration/MessageConfiguration.cs#L21


We're looking feedback on:

  1. Should sending a message or publishing a event to multiple destinations be supported? For the above example, the message would be published to both QueueA and QueueB?
  2. Alternatively if not supported, should configuration like the above be disallowed via an exception at startup? Or perhaps at least a LogWarning explaining the precedence.
martincostello commented 7 months ago

We used to have a feature like 1 in JustSaying, where we would publish one message to two regions as part of a (with hindsight over-complicated) disaster recovery scenario, hence used to.

At first sight, 2 feels counter intuitive as to how the MS DI container works by default i.e. last one wins.

If those were the only two options, 1 feels like the least surprising to me.

HEskandari commented 7 months ago

What you may need here, is to think bigger than messages. Conceptually 'messages' can be used as commands or events. Commands are sent to a destination to perform an action. There's always 1-1 mapping between the sender/recipient of a command. Events on the other hand signal an event that has already happened and can be sent to multiple recipients. The mapping between publisher of an event to subscribers are 1-many. Admittedly AWS messaging may not be the right abstraction to build this, but need to make it so that this is possible.

normj commented 7 months ago

@HEskandari I would expect the the 1 to many scenario be handled in the configuration of the AWS resources. For example if you need multiple consumers then the endpoint of the message type should be an SNS topic or EventBridge which can fan out the publishing of the message. It doesn't seem worth it to have the application logic itself hand the dispatching of the message to multiple endpoints.

ashovlin commented 5 months ago

Thanks for the feedback so far, we're going to leave this open for at least a few months longer to see if folks are still looking for 1-to-many support.