Consistency Tokens / Optimistic Concurrency

[FoseFx]

Describe the feature

Consistency Tokens are used to communicate the assumed state of a resource (i.e., aggregate). The last event processed on an aggregate can serve as such a Consistency Token.

Reads should carry consistency-tokens (i.e., event ids) for every aggregate involved. When a client attempts to write to an aggregate, they should be able to provide this token.

If the state has not changed since, the update is processed. Else, the client is provided with a more current state of the aggregate and its event id. As the relation between the services and EventStore (which is the source of truth) is only eventually-consistent, we are not able to guarantee this state is, in fact, the current state.

The frontend is can then resolve the conflict, wither automatically, or with help of the user.

Why is this feature needed?

Multiple clients may attempt to write to the same aggregate at the same time. Our endpoints generally only require the changed fields to be sent to the backend, which contains the impact (e.g., if Alice updates the name of a patient, and Bob the same patient's bed, no change will be overwritten). However, concurrent writes to the same field can of course still occur. Writes themselves are atomic, which means we will never end up in an invalid state, yet a slightly delayed write might overwrite a change that was just made. The damage is especially great with large free-text fields, for example. When two people are trying to add a new paragraph, only the last write "wins".

This problem is of course solved. The following alternative solutions have been considered:

• Conflict-free replicated data types
• Notify every potential writer about changes (and stream them in, if possible)

Currently, we do not assume any aggregate to suffer from high congestion, thus the solutions above don't seem to reasonable in terms of dev cost and maintenance.

Additional context

No response

[FoseFx]

Feedback @DasProffi?

[DasProffi]

This seems like a very reasonable approach, although it places the majority of the implementation burden on the Frontend, which doesn't have any generalized methods for form validation or request refetching to deal with conflicts yet. Both of which can be combined and should not be too difficult, but give me a few days to consider this further.

[MaxSchaefer]

@FoseFx can you and @DasProffi discuss the respective changes to the API schema and bring it to main

@DasProffi and I already discussed some frontend related topic for this implementation. If a request to an endpoint, for example UpdateTask resolves in a conflict, is it really necessary to return the diff? The frontend cannot really handle this response because the data to diff with comes from a projection.

[FoseFx]

Update:

Conflict detection will now happen on the backend. We will define a generic Conflict proto message, which will be part of the Update Responses. It should contain the real is state, and want state requested, along the attributes which were altered and the latest consistency token.