search

apache / pekko-persistence-cassandra

A replicated Apache Pekko Persistence journal backed by Apache Cassandra
https://pekko.apache.org/
Apache License 2.0
9 stars 10 forks source link

Feedback: ScyllaDB support and compatibility considerations. #135

Open hanishi opened 4 months ago

hanishi commented 4 months ago

When attempting to use the tables-autocreate feature specified in an application.conf file, I encountered a failure due to a compatibility issue between Cassandra and ScyllaDB. The problem arises because the unchecked_tombstone_compaction option, used within the compaction strategy configuration, is not recognized by ScyllaDB.

The plugin expects this option during the table creation process. The relevant code can be found here.

I worked around this by manually creating the necessary tables without using the unchecked_tombstone_compaction option. This workaround suggests that broader compatibility might be achievable with some adjustments. Given the growing popularity and adoption of ScyllaDB, it would be nice if future versions of the plugin could consider providing some support for ScyllaDB.

pjfanning commented 4 months ago

Would it be possible to tell ScyllaDB team about the missing support for this param? They seem to claim that they support tombstone compaction - eg https://github.com/scylladb/scylladb/issues/1487 and https://java-driver.docs.scylladb.com/scylla-3.11.0.x/api/com/datastax/driver/core/schemabuilder/TableOptions.CompactionOptions.html#uncheckedTombstoneCompaction-java.lang.Boolean-

pjfanning commented 4 months ago

also: https://lists.apache.org/api/plain?thread=16b01wk5tksg6646npl71n7fzck7o6mg

hanishi commented 4 months ago

Well, the tombstone compaction, by means of tombstone_threshold and tombstone_compaction_interval, is supported, but this specific option isn't I guess. 😩

ConfigurationException: Invalid compaction strategy options {{unchecked_tombstone_compaction, false}} for chosen strategy type
hanishi commented 4 months ago

@pjfanning let's find out. https://forum.scylladb.com/t/cassandras-unchecked-tombstone-compaction-not-recognized/1360?u=hanishi

pjfanning commented 4 months ago

@hanishi thanks for the details. They should be useful for other ScyllaDB users.

The docs say that the tables should be manually set up. https://pekko.apache.org/docs/pekko-persistence-cassandra/current/journal.html#schema

I think I can maybe come back and update the docs a bit but it take a few days.