Add a new storage adapter - nosql such as mongo, cassandra

[Johnmuchiri]

Description I would like to add a nosql adapter for eventually to leverage on the features that come with non-relational databases.

Could you please clarify the following as implemented in eventually-pg

1. The watermark and lease concept
2. The way you are managing competing consumers
3. actor and consumer concurrency.

[Rotorsoft]

Hey John, here is a quick summary of the "event" Store interface (port) and what is expected of new adapters:

On the query side, you need a way to access the event log using basic filters, there is nothing special here. There are also the seed and reset methods to initialize and reset the store, something you would use as part of your CI/CD migration or testing pipelines, but not as part of the primary operations.

On the write side, you have the commit method to append new events to a stream with optimistic concurrency (expectedVersion). The expected version is automatically provided by the framework when executing commands - the last loaded event that was used to reduce the state of the executing command is expected... unless the user provides an explicit version in cases when you need to control this from the GUI for example. You are trying to avoid concurrent users having race conditions on the same stream while executing concurrent commands on it. One extra feature you might want to have is the ability to signal subscribers when new events are added to the store, to avoid unnecessary polling when you have real-time requirements.

On the consuming side, there is the concept of leasing the stream. Here you are serializing access to consumers. The poll method locks access to polling while the consumer is processing events and ack for completion. There is a timeout in case the consumer cannot complete/ack. Note that there is a consumer id in these operations, something like a consumer group, and each id stores the position of that group in the stream (watermark), so you can have multiple groups consuming from the store concurrently.

Now, this is the first iteration, with the goal to keep it simple and provide a practical interface to policies and projectors to consume events from the "all stream", with basic batch processing (for projections). At some point in the future, we can explore other options to consume from specific streams within the "all stream", similar to the query filters (like a partitioning strategy), and optimize this interface using read-ahead buffering/caching, etc.

Hope it helps!