[Feat] Add Dead-Letter Queue (DLQ) for Failures

moov-io / achgateway

Payment gateway enabling automated ACH operations in a distributed and fault tolerant way.

https://moov-io.github.io/achgateway/

Apache License 2.0

52 stars 19 forks source link

[Feat] Add Dead-Letter Queue (DLQ) for Failures #139

Open yordis opened 1 year ago

yordis commented 1 year ago

Along with acking the message to avoid queueing the system on a failure, it is also prudent that we must publish successfully into a DLQ topic the failed message.

Acceptence Criteria

As an operator:

I would allow to configure a DLQ topic name where the failed messages will be published to

As an application dev:

I expect a DLQ message to be published when a message is processed.

adamdecaf commented 1 year ago

This seems like a handy feature. Would you like to work on it?

adamdecaf commented 1 year ago

I'm assuming failures encountered by https://github.com/moov-io/achgateway/pull/136 would appear on this DLQ, right?

yordis commented 1 year ago

Yes, everything that could be "missed" in the processing would be a DLQ for late consumption of the message.

Being said,

You could also split the DLQ into broken message structures and have a failures topic for those things.

I instead do the latter architecturally.

yordis commented 1 year ago

This seems like a handy feature. Would you like to work on it?

I will as soon as I have some time, this will be critical to us