Support for SQS FIFO Queues

[brainmurphy]

When FIFO Queues were first introduced by AWS, subscribing to an SNS Topic wasn't supported, which made them less useful for JustSaying. Happily, in 2020, AWS added this support, so we should now think about adding this functionality to the library.

I can think of three parts to this: the infrastructure, the publishing API/s and the receiving API.

In terms of infrastructure, can we simply delegate this to the "optional infrastructure" work - i.e. allow people to manage it using their favourite tool (CloudFormation, Terraform, etc)? All major Infrastructure as Code products allow creation of FIFO Queues and Topics at this point. The other alternative is to modify JustSaying's infrastructure creation code to support the additional configuration that FIFO Queues and Topics need: attributes such as deduplication scope, content-based deduplication, etc. What would be the best way to specify those additional attributes with the JustSaying API?

JustSaying, of course, supports publishing messages to both SNS Topics and directly to SQS Queues. When publishing messages to an SNS FIFO Topic, there are additional parameters to control the deduplication - MessageDeduplicationId and MessageGroupId. Similarly, when sending to an SQS FIFO Queue we can specify MessageDeduplicationId and MessageGroupId. We would need to think about the best way of flowing these new parameters into the AWS API calls while keeping the JustSaying API as the abstraction it currently is.

References: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_CreateQueue.html https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html

https://docs.aws.amazon.com/sns/latest/api/API_CreateTopic.html https://docs.aws.amazon.com/sns/latest/api/API_Publish.html

[stale[bot]]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[PritamSanganiTW]

Hi @brainmurphy @martincostello - is there an update on when FIFO SQS queues and SNS topics will be supported by this library?

[PritamSanganiTW]

additionally to what is mentioned in the description, there would need to be changes to how the error queue name is generated because currently _error is prefixed to the end of the queue name and so if some-queue.fifo was specified as the queue name, the error queue would currently be called some-queue.fifo_error which is not valid. This logic would need to be updated or the configuration builder would need to be extended to allow specifying the error queue name.

[PritamSanganiTW]

Hi @martincostello I think you were on holiday last week so may have missed this message, so bumping for visibility.

[martincostello]

We're not currently actively planning to add this functionality. We'd be open to community members contributing such functionality to add to a future version.