search

Eventuous / eventuous

Event Sourcing library for .NET
https://eventuous.dev
Apache License 2.0
442 stars 70 forks source link

SQL Server Subscription skips an event on non-initial startup #291

Closed PehrGit closed 9 months ago

PehrGit commented 9 months ago

Describe the bug When we have some events in the database and so the checkpoint for that subscription is not NULL, if we start the application and add 2 events, the first of those is not retrieved and thus not processed by the projections. For our case specifically, we have an Add and an Update event, which are both produced by a single command. When we start the application and handle this command, the Add is not processed and then the Update tries to edit a record which does not exist.

To Reproduce

  1. Have SQL Server as the event store.
  2. Have SQL Server as the projection db.
  3. Start application
  4. Produce events 1 and 2
  5. Stop application
  6. Start application
  7. Produce events 3 and 4
  8. See error

Expected behavior Events 3 and 4 are sequentially retrieved from the db and processed by the projections.

Screenshots N/A

Desktop (please complete the following information):

Additional context We believe we have found the problem and will create a PR shortly.

In SqlServerSubscriptionBase we see the following Subscribe https://github.com/Eventuous/eventuous/blob/a66fb591bb82c68fcba172a3f4ef26f073dedff0/src/SqlServer/src/Eventuous.SqlServer/Subscriptions/SqlServerSubscriptionBase.cs#L37-L43

It gets the current checkpoint, adds 1 to it and calls the PollingQuery https://github.com/Eventuous/eventuous/blob/a66fb591bb82c68fcba172a3f4ef26f073dedff0/src/SqlServer/src/Eventuous.SqlServer/Subscriptions/SqlServerSubscriptionBase.cs#L63-L72

Which receives the <checkpoint + 1> value as position and then calls the PrepareCommand in the SqlServerAllStreamSubscription (the same happens in SqlServerStreamSubscription) : https://github.com/Eventuous/eventuous/blob/a66fb591bb82c68fcba172a3f4ef26f073dedff0/src/SqlServer/src/Eventuous.SqlServer/Subscriptions/SqlServerAllStreamSubscription.cs#L25-L28

This takes the <checkpoint + 1> value, adds 1 to it again, and passes <checkpoint + 2> to the stored procedure ReadAllForwards https://github.com/Eventuous/eventuous/blob/a66fb591bb82c68fcba172a3f4ef26f073dedff0/src/SqlServer/src/Eventuous.SqlServer/Scripts/ReadAllForwards.sql#L1-L15

Where all events are retrieved whose GlobalPosition value is greater than the incoming <checkpoint + 2> value.

We believe that adding 1 to the position makes sense because you only want to get events which are ahead of the checkpoint. But it seems doing a +1 twice is causing it to retrieve events which are ahead of the checkpoint by 2 or more.

We see that the same code flow is present in the Postgres implementation so this might suffer from the same problem: https://github.com/Eventuous/eventuous/blob/a66fb591bb82c68fcba172a3f4ef26f073dedff0/src/Postgres/src/Eventuous.Postgresql/Subscriptions/PostgresSubscriptionBase.cs#L34-L34

https://github.com/Eventuous/eventuous/blob/a66fb591bb82c68fcba172a3f4ef26f073dedff0/src/Postgres/src/Eventuous.Postgresql/Subscriptions/PostgresAllStreamSubscription.cs#L27-L27