Using scylla-sstable on sstables, runs the risk of the sstable disappearing by the time scylla-sstable tries to read it, because ScyllaDB decides to compact it at the wrong moment. Scylla-sstable also reads the schema-table stables, which runs the same risk. For this reason tests using scylla-sstable indirectly are often found to be flaky, occasinally failing because an sstable was pulled from under the feet of scylla-sstable.
We currently rely on the callers to disable compaction on the system_schema keyspace and the examined user keyspace for the scope of invoking scylla-sstable. Callers however don't always remember to do this and we keep finding cases where this is missing.
I think the best place to do this would be in scylla-ccm, inside the methods which invoke scylla-sstable. Users would not have to to remember to disable compaction, they would automatically be safe.
denesb
commented
9 months ago
Using scylla-sstable on sstables, runs the risk of the sstable disappearing by the time scylla-sstable tries to read it, because ScyllaDB decides to compact it at the wrong moment. Scylla-sstable also reads the schema-table stables, which runs the same risk. For this reason tests using scylla-sstable indirectly are often found to be flaky, occasinally failing because an sstable was pulled from under the feet of scylla-sstable. We currently rely on the callers to disable compaction on the system_schema keyspace and the examined user keyspace for the scope of invoking scylla-sstable. Callers however don't always remember to do this and we keep finding cases where this is missing.
I think the best place to do this would be in scylla-ccm, inside the methods which invoke scylla-sstable. Users would not have to to remember to disable compaction, they would automatically be safe.