Open utterances-bot opened 2 years ago
Great write-up! I have one concern. What about the frequency of the outbox polling/checking. Imagine that I am currently executing something, and I require that the message is pushed onto the queue immediately - of course - if the queue is available.
Is there a way to say "Send now - otherwise store in the outbox" has these been thought about or implemented in the outbox implementation?
So if I can send it immediately, this would be the preferred option. Otherwise, it would go to an outbox and get processed and retried up to the threshold. If it ends up in the outbox, I have no option but to wait for it to be processed when the queue becomes available.
Thanks for a great write-up!
Hey @iangregsondev! 👋 Thank you for the comment! Valid concern! The current implementation introduces a delay between the outbox sending time and the message delivery time, as you correctly point out.
The benefit of the outbox pattern lies in executing multiple operations as a unit of work in one atomic transaction which then guarantees the "at least once delivery". So "either everything or nothing is stored in the database". For this reason, we have a strict rule when implementing the outbox pattern; We must store the message in the database before attempting to deliver the message in the queue.
Yes we can! 💪
Once we commit the transaction, we can follow the below steps using a targeted processing method
If the outbox pattern is implemented in a custom fashion without using a generic-purpose outbox library, all good. We just need to call the above code after we commit the transaction 🎉
If we want to provide this functionality in a general-purpose library, I can see the following options:
publisher.Send
method. Once the method is called, the SQL transaction is executed, and if successful, we attempt to deliver the message following the steps mentioned above. This could be another method of the publisher that could be used in a case-by-case scenario.Hope this helps! 🙏
Implementing the Outbox pattern in go | pkritiotis Software Engineering
This article presents the design and implementation of the outbox pattern in go.
https://pkritiotis.io/outbox-pattern-in-go/