cloudstateio / cloudstate

Distributed State Management for Serverless
https://cloudstate.io
Apache License 2.0
763 stars 97 forks source link

Projections for value entities #498

Open jroper opened 3 years ago

jroper commented 3 years ago

The current event log support only supports event sourced entities, which means we can only do projections for event sourced entities. We could support projections for value entities, by using a last updated time and/or last updated sequence number column in the underlying store, that gets polled. The semantics of this would be that not every update for every entity was guaranteed to be delivered, but the last update for each entity was guaranteed to be delivered - that is, if you stopped updating entities, you would be guaranteed that eventually, the most recent version of every entity will have been the last version of that entity that was passed to the read side.

Some implementation notes:

Spanner

This can be implemented in Spanner using the commit timestamp, with the polling implemented in the same way that the akka-persistence-spanner eventsByTag query is implemented.

Postgres

A timestamp column could be used, or a sequence number in combination with a timestamp column. In either case, I believe we would need a small consistency delay on reading, since the time between when the timestamp is generated or next sequence number is returned, and when the transaction is committed (and therefore visible) would be non zero, allowing for out of order insertions and therefore reads of the timestamp/sequence number.

Cassandra

A time based UUID column cloud be used. An eventual consistency delay, similar to that used by akka-persistence-cassandra events by tag query, would be needed to ensure in order reads.