search

dotnet-architecture / eShopOnContainers

Cross-platform .NET sample microservices and container based application that runs on Linux Windows and macOS. Powered by .NET 7, Docker Containers and Azure Kubernetes Services. Supports Visual Studio, VS for Mac and CLI based environments with Docker CLI, dotnet CLI, VS Code or any other code editor. Moved to https://github.com/dotnet/eShop.
https://dot.net/architecture
24.53k stars 10.36k forks source link

Ensuring Events are Published in the LocationService #619

Closed fauxcoding closed 6 years ago

fauxcoding commented 6 years ago

I noticed someone asked about ensuring events are published (Issue: 279) and became apparent that some of the services are using the Outbox Pattern, which is awesome.

However, I noticed that the outbox pattern wasn't used within the LocationService, which is using MongoDB under the hood. I understand that it would be difficult to do the outbox pattern with Mongodb given its lack of ACID transactions (version 4 should solve that).

In the meantime, what do you recommend the approach should be for the current LocationService to ensure events are being published?

mvelosop commented 6 years ago

@Hi @heymega, @CESARDELATORRE, thinking about this and an old related issue (#500), I was wondering, if it's ok to do this: When a service subscribes to some event queue upon starting up, specify a date filter to include possible "cold events" it might have not received.

So, if it's the first subscription ever to the queue, then it seems ok to ignore previous events, but for the subsequent service startups (re-subscriptions) it should receive all relevant events after the last one previously received.

This would even handle failures on the outbox pattern.

Would this be a good pattern to handle this scenario?

fauxcoding commented 6 years ago

I was more concerned with whether the event actually gets published, not so much about the consuming of the event itself.

The location service currently does this...

  1. Saves the location to a mongodb location document
  2. Publishes an message to the topic

The problem here is that number 1 could succeed and number 2 could fail.

If we were to refactor this logic so that it followed the Outbox pattern we might do the following...

  1. Saves the location to a mongodb location document
  2. Saves the event to a mongodb event document
  3. A background task checks the event document and publishes events to the topic

I'd imagine that this scenario only works for databases which support ACID transactions. Number 1 and number 2 need to be in a transaction. However, mongodb doesn't currently support this.

I guess the Location Service could be modified to do two phase commit but I feel like you would need to focus a lot of effort on compensating steps in case something went wrong.

I'd be curious, for my own learning, if 2PC or another method could be implemented to improve this service.

mvelosop commented 6 years ago

Hi @heymega, I think I understand your issue better now.

And the final answer is a definitive "it depends", so 2PC could be a good solution or not depending on the business needs.

The Location service was meant to be a very simple one, with no advanced requirements, so that was deemed enough by then.

I suggest you read @CESARDELATORRE's detailed explanation on the subject in issue #501.

Hope that helps.

fauxcoding commented 6 years ago

Oops I didn't realise someone had already asked this. I think I need to work on my Git issue searching!

Thank you for your help 👍