eventflow / EventFlow

Async/await first CQRS+ES and DDD framework for .NET
https://geteventflow.net
Other
2.38k stars 446 forks source link

Concurrency and Container based Services #405

Closed mattgi closed 6 years ago

mattgi commented 6 years ago

In considering just the write side of the equation, I am wondering how one deals with concurrency across services.. also hoping for simple guidance on how to structure/host EventFlow across multiple domain driven micro services.

For the purpose of the question, here is what I assume to be a common requirement:

untitled diagram 3

Two micro services, separated by bounded contexts.. Item Service deals with Items, User Service deals with Users. An Event in one may impact another (e.g. Deleting a User would mean the Item would also need to reflect that it no longer has an Owner, etc.).

Questions:

  1. Item Service is scaled to 2 nodes/containers. How do I ensure the same container manages the same Source Ids (and is this the recommended approach to dealing with concurrency issues faced by scaling)?

A thought is that perhaps the Command Bus should be externalised to better manage the routing.. so then all requests hit a single API Gateway which then creates Commands and pushes them to a Distributed Command Bus. The bus then invokes handlers and has some smarts about maintaining source ids or queues things... but is this a good approach, how could it work, what communication mechanism would you use, and how would you host the handlers)?

  1. Services that communicate via Sagas (i.e. Service 1 Event initiates Service 2 Command). Does this just work once you utilise RabbitMQ or similar as an Event Bus? I guess the Event Bus would require the same smarts as a distributed command bus if Service 2 was running on multiple nodes.

Are people typically running multiple instances of the same service in production? Am really curious on the architecture and design of full, production hardened, implementations of EventFlow to be sure I'm heading down the right path.

rasmus commented 6 years ago

Hi

EventFlow handles concurrency using optimistic concurrency within each aggregate root. This is done using the aggregate sequence numbers of the aggregate events. If a collision is detected, the command is retried on the updated aggregate root as it might still be a valid operation, e.g. the order of changing title and sub-title of a book doesn't really matter.

As for handling concurrency for two aggregates, between bounded context or even within the same instance is something else. We typically try to design our applications in a way that removes the need.

The typical text book example would be something like this

With hard concurrency 1) User A and B has 100$ in their shared account 2.a) User A pays a bill for 100$ 2.b) At the same time user B pays another bill using the same account for 80$ 2) One of the users is denied

Alternative 1) User A and B has 100$ in their shared account 2.a) User A pays a bill for 100$ 2.b) At the same time user B pays another bill using the same account for 80$ 3) Users A and B now have -60$ on their account and back starts to collect interests

I know its a very basic example, but I hope it clarifies it up a bit. In your case deleting a user will ripple through different systems. Consider if this really needs to be instantaneous.

A few other notes

Hope I got it all

mattgi commented 6 years ago

Awesome, thanks @rasmus. Our problem space is a simple one, but well suited to event sourcing. Few struggles in best understanding how to deal with concurrency but am getting there.. loving EventFlow so far - just wish there was more doco or examples to work from.. both tedious and time consuming to maintain so understand why things are the way they are. Thanks for taking the time to respond in such detail though, appreciate it.