Reliable Read Model Updates

[Jejuni]

Playing around with the library I noticed that when using EntityFramework for the readmodel and store any exception thrown during the insertion/update of the readmodel will result in the event being committed to the store but the readmodel update being dropped (due to the exception).

I tried to solve this by having my readmodel context and eventstore context both use the same connection and transaction. However, after having a look at the source code, the context is disposed after every operation for which it is created via the IDbContextProvider. This also disposes the underlying DbConnection.

My questions are:

• Why is the EventFlow Entity Framework Infrastructure disposing an object for which it was not in charge of creating?
• Was it a deliberate design choice to have the event store updates in a separate transaction as the readmodel updates? And if so, how would one go about updating the readmodels reliably, considering the readmodel updates are triggered synchronously after the event store update?

Thanks!

[rasmus]

Hi @Jejuni I think you hit issue #439.

Generally EventFlow tries to model everything around an eventually consistent model as doing everything in a transaction would create a lot of locking. We host on of the largest sites in Denmark and with the amount of throughput our event store has, doing transactions that span multiple read model isn't durable (we have quite a few). Most of our read models are hosted in separate micro services, to which we send updates using RabbitMQ. You could say that EventFlow tries to use the same model within a single process.

But, issue #439 is still very relevant as you should be able to rely in the updates happen at least once.

[rasmus]

Maybe have a look at https://blog.sapiensworks.com/post/2014/06/04/Unit-Of-Work-is-the-new-Singleton.aspx

[Jejuni]

Hey,

I totally agree that having things in a singular transaction is a really bad idea for scalability. I'm in the process of evaluating Eventflow with MassTransit (Asynchronous messaging framework) to use in our internal application and I really love the way the DDD concepts are implemented.

My first idea was to update the readmodels asynchronously as well (even the ones residing inside the same service) by just dispatching a corresponding integration event to RabbitMq and then doing the updates in reaction to that. For small read model updates the approach of using EventFlow's readmodel update out-of-the-box would've been awesome to use as well, though.

Although I'd have to disagree with updating in process being the same as updating via RabbitMq. With RabbitMq (or any bus) you get retries until you ACK the message. So if you're idempotent you should be fine even if 1 out of 4 readmodel updates failed. Simply being in process means that an exception at the same point in your code leaves your application in a permanently inconsistent state, depending on circumstances.

We host on of the largest sites in Denmark and with the amount of throughput our event store has, doing transactions that span multiple read model isn't durable (we have quite a few). Most of our read models are hosted in separate micro services, to which we send updates using RabbitMQ

This is the kind of approach I'd now be going with as well. As a follow-up question, since you're using EventFlow at scale in a large app already: Do you use the built-in capabilities of EventFlow's auto-read model updates at all? Or do you separate out all readmodel updates using messages over rabbitmq?

[rasmus]

We on have some very basic read models within the service that uses EventFlow since is very isolated to creating aggregates. In most of our other micro services we don't use EventFlow but have some custom code to update read models based on messages from RabbitMQ. RabbitMQ is used for a lot of other events as well.

[Jejuni]

Thanks for your answer!

[GreatGodJackChen]

@Jejuni 通过EventFlow+MassTransit 实现单独读模型服务实现了吗，非常希望和你沟通交流