Scylla does not start up if data directory has temporary Cassandra sstables

[penberg]

I ran cassandra-stress against Cassandra 2.1.8, issued nodetool flush, stopped Cassandra, and copied over the sstables (including system schema) to Scylla:

$ cp -R ~/testing/apache-cassandra-2.1.8/data/data/keyspace1/ node1/data/
$ cp -R ~/testing/apache-cassandra-2.1.8/data/data/system/schema_* node1/data/system/

Scylla, however, refuses to boot because it trips over temporary sstables:

INFO  2016-03-30 14:26:18,355 [shard 0] database - Keyspace node1/data/keyspace1: Reading CF standard1
WARNING: exceptional future ignored of type 'sstables::malformed_sstable_exception': invalid version for file keyspace1-standard1-tmp-ka-14-Data.db. Name doesn't match any known version.
WARNING: exceptional future ignored of type 'sstables::malformed_sstable_exception': invalid version for file keyspace1-standard1-tmp-ka-14-Data.db. Name doesn't match any known version.
WARNING: exceptional future ignored of type 'sstables::malformed_sstable_exception': invalid version for file keyspace1-standard1-tmp-ka-14-Data.db. Name doesn't match any known version.
Exiting on unhandled exception of type 'sstables::malformed_sstable_exception': invalid version for file keyspace1-standard1-tmp-ka-14-Data.db. Name doesn't match any known version.

@avikivity says that we should just ignore them but I did not verify that it's what Cassandra does.

[avikivity]

Perhaps, instead of ad-hoc pattern matching, the sstable code should register patterns and handlers for those patterns. It could then register empty handlers for these patterns.

[tzach]

@penberg is it a migration from Cassandra issue, or can it happen with just with Scylla?

[avikivity]

C* only.

[tzach]

@raphaelsc any update?

[raphaelsc]

@tzach Not yet. I will take a look at it soon.

[slivne]

@raphaelsc I think we have merged a fix to diregard unknown components - does it cover this as well ?

[raphaelsc]

On Wed, Nov 9, 2016 at 9:47 AM, Shlomi Livne notifications@github.com wrote:

@raphaelsc https://github.com/raphaelsc I think we have merged a fix to diregard unknown components - does it cover this as well ?

No. This problem is that scylla is unable to recognize a temporary file generated by Cassandra. Cassandra puts 'tmp' right before version, and Scylla thinks 'tmp' is a version.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/scylladb/scylla/issues/1148#issuecomment-259396523, or mute the thread https://github.com/notifications/unsubscribe-auth/ABWAcxvRzhwmJbrhntlhHN-uRhAl2_OTks5q8bK9gaJpZM4H7xK4 .

[raphaelsc]

On Wed, Nov 9, 2016 at 6:48 PM, Raphael Carvalho raphaelsc@scylladb.com wrote:

On Wed, Nov 9, 2016 at 9:47 AM, Shlomi Livne notifications@github.com wrote:

@raphaelsc https://github.com/raphaelsc I think we have merged a fix to diregard unknown components - does it cover this as well ?

No. This problem is that scylla is unable to recognize a temporary file generated by Cassandra. Cassandra puts 'tmp' right before version, and Scylla thinks 'tmp' is a version.

Just checked that scylla will actually be unable to boot if it finds any file that doesn't match the component pattern. I think we should only ignore those files. There's no need to specially handle useless temporary files coming from C*.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/scylladb/scylla/issues/1148#issuecomment-259396523, or mute the thread https://github.com/notifications/unsubscribe-auth/ABWAcxvRzhwmJbrhntlhHN-uRhAl2_OTks5q8bK9gaJpZM4H7xK4 .

[avikivity]

But, if there is a .tmp file from C*, won't there also be other component files that we should ignore?

If we ignore the .tmp file, we won't ignore these components.

[bhalevy]

@raphaelsc is this bug still valid?