Cassandra storage backend

[krasserm]

This is a resolution to the suspended #59 which will also free us from maintaining a separate

• event log archive (as events can be permanently stored in Cassandra and scaling with data/event volume can be done by adding further nodes)
• event log index (as it can be stored in Cassandra as well). The index is needed to index events by aggregate id.

This will for sure reduce operational complexity but may also reduce development effort. We will be able to re-use a lot of work done in akka-persistence-cassandra which also has several production deployments.

Regarding the discussion about read consistency in akka-persistence-cassandra, we need to consider that batch writes must not cross partition boundaries. If a new batch write doesn't fit into the remaining space of a partition, the partition will be padded with placeholders (consuming sequence numbers) and the batch will then be written to the next partition. Consequence is that sequence numbers seen by the application are not contiguous which is fine in our case. Having this implemented, we will be able to guarantee the event replication semantics required by Eventuate.

Another significant advantage is that all writes to Cassandra are idempotent (in contrast to writes to Kafka) which simplifies event log actor development.

[giena]

What about the caveats here: https://github.com/krasserm/akka-persistence-cassandra#caveats A post that has disappeared: https://support.datastax.com/entries/69902949-ReadTimeoutException-seen-when-using-the-java-driver-caused-by-excessive-tombstones And an analogy with queue implementation in Cassandra: http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets I've experienced these issues when i test the akka persistence plugin for Cassandra at a high throughput. I do a snapshot at 50000 msgs, then i delete the 50000 messages. I've hundreds of persisted Actor. Cassandra works for some minutes and then i get ReadTimeOutException. On the Cassandra server side i've too many tombstones... IMO Cassandra is not the best solution since we can do a lot of deletes when we do event-sourcing. The best usage with Cassandra (like Aerospike) is to store long life data with a lot of updates. The Kafka use case is to journalize so it is best suited for event sourcing. The deletion does not cost with Kafka. But perhaps you have some workarounds for these Cassandra caveats ?

[newca12]

Google cache to the rescue : http://webcache.googleusercontent.com/search?q=cache:hhmnz38DpW0J:https://support.datastax.com/mobile/switch%3Freturn_to%3Dhttps%253A%252F%252Fsupport.datastax.com%252Fentries%252F69902949-ReadTimeoutException-seen-when-using-the-java-driver-caused-by-excessive-tombstones+&cd=1&hl=fr&ct=clnk&gl=fr

[krasserm]

@giena @newca12 Eventuate uses Cassandra for long-term event storage and event deletion will not be part of the public API. Disks are cheap and events are stored in a way that the Cluster can scale with data volume just by adding nodes. Consequently, all concerns regarding queue anti-patterns do not apply here. If deletion should be necessary at all, an appropriate administrative tool will be provided. The reasons why we decided against using Kafka are explained in #59.