search

cosmicpython / book

A Book about Pythonic Application Architecture Patterns for Managing Complexity. Cosmos is the Opposite of Chaos you see. O'R. wouldn't actually let us call it "Cosmic Python" tho.
https://www.cosmicpython.com
Other
3.39k stars 527 forks source link

Making the message bus reliable #332

Open Arseniy-Popov opened 2 years ago

Arseniy-Popov commented 2 years ago

I wonder if the message bus used in production at MADE is different from the one described in the book in ways which make it more reliable. In the scenario of a batch quanity change deallocating a number of order lines, if the service dies after a batch quantity has been changed, but before the order lines have been allocated again, they are going to end up unallocated as events are not persisted.

One way of rectifying this that I see is writing events to a message broker, perhaps using a transactional outbox, to a channel internal to the inventory service, to be later consumed by the same service. Another way is writing events to a database and having a poller querying the database and consuming events from it as one of the entrypoints into the service.

Both this options seem to nullify the need for the message bus by replacing it either with a message broker or a database as the place where events reside inbetween being processed. So I wonder if these approaches are feasible at all and what your thinking is on how an in-app and a persistent message buses compare.

If someone apart from the maintainers wants to chime in on this, please do!

dbaber commented 6 months ago

I think in the Epilogue they talk about this and they use EventStore.

georgek commented 4 months ago

Why not process the whole message bus in a single unit of work?

hjwp commented 4 months ago

it depends what you want really. after you finish processing one event, you have to decide whether you want to commit and persist your changes, or if you want to move on to provessing all the other events that may have been generated (and the ones they generate, and the ones they generate...), and then decide if it's ok that an error in one of those downstream events causes your whole transaction to be rolled back. it really depends on how independent they handling of each event needs to be, from a business perspective.