Event sourcing candidate

[olafura]

Hi we are looking at good ways of doing event sourcing in Elixir and we have been looking at: https://github.com/slashdotdash/eventstore

It has some problems with clustering setup and it might be too specialized where we can't just easily add new event handlers as we would get with using a general solution solutions for the log like with Kafka.

I'm just wondering if Antidote would be a good candidate to do most of the hard work in doing event sourcing in your option?

Also is it crazy to use Antidote at this stage, it seems to be build on pretty solid foundations and we are not expecting it to be bug free, but should we wait to explore it?

[bieniusa]

Hi, thanks for getting in contact! I am afraid that my knowledge on event sourcing is quite limited, so I am not sure what requirements you actually have. If you can provide more information (availability, scalability, general setup, ...), we can try to see what is possible with Antidote.

We have a few companies that use Antidote for prototyping and we are working hard on getting it production-ready. If you want to give it a try, we would be glad to help getting it into shape for you :)

[olafura]

Hi, my knowledge of event sourcing is also not going to be the best since I'm new to it. Here is an image

But what is drawing us to that concept is:

• Immutability (Having a clear record of what was asked when)
• Separation of asking for something to to be done (command) and it being executed by a event handler to be possible being replied to a subscriber.
• We have a command that is evaluated by a command handler that might be turned into an event that an event handler is going to react to. So these things are going to be stored in a log (for example Kafka) for persistence. So you have both what was requested and what was the consequence.
• Replaying things if something goes wrong.

We aren't going to have large scalability problems yet and we are moving from isolation (deis) to a cluster setup so we can use Erlang a bit better. We are currently having problems with the version in a stream because they require having a singleton handling it and if you make other database connections you get a warning about the version number.

Here is probably better explanations about Event Sourcing. http://cqrs.nu/Faq

Why we like Antidote:

• It's managing very easy to reason about structures.
• It has subscriptions.
• It does distributed computing.
• It's on the BEAM.
• It's using proven technology underneath (riak).
• I was going to mention that you can access it outside of the BEAM but most databases and other solutions have that also :)

[boxxxie]

We would need a durable log, where ordering of events is stable. We would want to read events from the log in an ordered fashion from any point in the log (from the beginning will be a common use-case) to the end, and then continue writing and reading from the end

[bieniusa]

Sorry for the delay, but I checked with some teammates regarding your requirements.

• It is certainly possible to subscribe to events in Antidote; we use ZeroMQ as one components.
• We are building on riak-core, but we do not use riak as backend.
• All items in the log are stored in a sequence; the backend is in the current version an append-only log. All updates on one data shard are ordered sequentially, and this order is the same for all replicas of this shard. Across shards we use vector clocks to relate the order of updates.

Antidote strength is actually with concurrent modifications of data types; but you can certainly also use it for immutable data. We currently do not have an API for getting data directly from the log; instead we serve locally available versions of a data item. So for your use case, we would need to adapt the interface to serve also the log(s) for the different shards, right?

[preguica]

Let me add just one thing to what Annette has already replied. Regarding subscribing events in Antidote, we have already done it for a project where clients would subscribe for the updates in a set of objects. The API for doing this is experimental, but it should be possible to use it.

Nuno

On Thu, Apr 6, 2017 at 4:08 PM, Annette Bieniusa notifications@github.com wrote:

Sorry for the delay, but I checked with some teammates regarding your requirements.

• It is certainly possible to subscribe to events in Antidote; we use ZeroMQ as one components.
• We are building on riak-core, but we do not use riak as backend.
• All items in the log are stored in a sequence; the backend is in the current version an append-only log. All updates on one data shard are ordered sequentially, and this order is the same for all replicas of this shard. Across shards we use vector clocks to relate the order of updates.

Antidote strength is actually with concurrent modifications of data types; but you can certainly also use it for immutable data. We currently do not have an API for getting data directly from the log; instead we serve locally available versions of a data item. So for your use case, we would need to adapt the interface to serve also the log(s) for the different shards, right?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SyncFree/antidote/issues/290#issuecomment-292100521, or mute the thread https://github.com/notifications/unsubscribe-auth/AJZR8vro0V12rqMh8tNJZUQkcHE4K5DDks5rtJ2QgaJpZM4MzLk3 .

[boxxxie]

can you point me to where i should be looking? examples?