get-eventually / eventually-rs

Event Sourcing for Rust
MIT License
558 stars 30 forks source link

Feature: Opinionated persistence and exactly-once storage guarantee #222

Open taqtiqa-mark opened 2 years ago

taqtiqa-mark commented 2 years ago

Is your feature request related to a problem? Please describe.

Guaranteeing that a message (state data) is persisted exactly once.

Describe the solution you'd like A generic preexisting solution that does not involve having to write server/client code implementing the guarantee .... NATS-JetStream

Describe alternatives you've considered Reimplementing NATS-JS functionality inside eventually and its storage adapters.

Additional context The idea is that eventually only support delivering aggregate data to a persistence store that can guarantee an 'exactly-once' quality of service.

While there are other options, NATS-JS does appear to offer the simplest starting point. Having delivered the data to a time and space bounded persistence store, say NATS-JS, there is the question of persisting it for times and in volumes that exceed that initial store capacities. As best I can tell, every user will come to this point regardless of what persistence options eventually offers.

There is an additional benefit in terms of interfacing with a classical SQL store. Specifically, there are NAT-JS clients in multiple languages that provide the much of the difficult implementation required to write an 'exactly-once' consumer that populates, say PostgreSQL. This relieves eventually of the burden of a providing 'exactly-once' storage adapters.

An additional (albeit small?) advantage of specializing to NATS-JS would be a simplification of the eventually code by allowing the storage detail to move from a generic parameter to an associated type?

I'll note this wall of text was prompted by a gnarly couple of days trying to make what I thought would be a simple refactor in the example app, where the compiler introduced me to aE0277 error with the suggestion the path out was by adding a self referential trait bound - having spent some time wrestling with the compiler over the repository and store I did wonder if it be simpler just to have the store as an associated type. That could be all wrong but it explains what prompted this feature request :)

Thanks for all your effort that you have put into eventually - much appreciated.

ar3s3ru commented 1 year ago

Hey @taqtiqa-mark, thank you for opening the issue and so very sorry it took me so much time to answer 🙇🏻

Just to make sure I understand, the point from your side would be:

  1. Do not use a traditional ACID database to store the aggregate state,
  2. Use a messaging system like NATS and the JetStream capability on top of it as an aggregate store,
  3. Use an ACID database as a "sink database" from the data on JetStream.

Did I get that right?