Open rookie7799 opened 4 years ago
fruch
commented
4 years ago nodetool refresh ? https://docs.scylladb.com/operating-scylla/nodetool-commands/refresh/fruch
commented
4 years ago can you share the /etc/scylla/scylla.yaml you are using ?
fruch
commented
4 years ago @psarna @elcallio, any idea what we are missing here ?
rookie7799
commented
4 years ago can you share the
/etc/scylla/scylla.yamlyou are using ?
Added to the main description.
rookie7799
commented
4 years ago
- Can you share example of the schema being used ? and a sample of the data ?
- Did you tried
nodetool refresh? https://docs.scylladb.com/operating-scylla/nodetool-commands/refresh/
CREATE KEYSPACE ase WITH replication = {'class': 'NetworkTopologyStrategy', 'us-east': '1' } AND durable_writes = false;
CREATE TABLE ase."2019_08_13_twitter" (
profile_id bigint,
affinity_id int,
PRIMARY KEY (profile_id, affinity_id)
) WITH CLUSTERING ORDER BY (affinity_id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'sstable_compression':'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND speculative_retry = '99PERCENTILE';
insert into ase."2019_08_13_twitter" (profile_id,affinity_id) values (1,1);About to try nodetool refresh.
rookie7799
commented
4 years ago Copied aa-*.db files over into ./upload/ dir and ran:
nodetool refresh -- ase 2019_08_13_twitter
nodetool: Scylla API server HTTP POST to URL '/storage_service/sstables/ase' failed: Failed to load new sstables: sstables::malformed_sstable_exception (invalid version for file aa-875-bti-CompressionInfo.db. Name doesn't match any known version.)
I think we can only read big format tables
https://stackoverflow.com/questions/58870769/better-understand-sstables-formats-big-vs-bti
from which version are you copying this data ? a datastax version ? or open-source cassandra 3.11 ?
rookie7799
commented
4 years ago I think we can only read
bigformat tables https://stackoverflow.com/questions/58870769/better-understand-sstables-formats-big-vs-btifrom which version are you copying this data ? a datastax version ? or open-source cassandra 3.11 ?
Datastax 6.5
fruch
commented
4 years ago I'll can check tomorrow with more people, but I think we can't read Datastax proprietary format directly. and need to migrate using something like https://github.com/scylladb/scylla-migrator while both are cluster are up and running.
fruch
commented
4 years ago try using datastax own tool, to create a "mc" "big" format sstables: https://docs.datastax.com/en/dse/6.0/dse-dev/datastax_enterprise/tools/toolsSStables/ToolsSSTabledowngrade.html
rookie7799
commented
4 years ago I think we can try the migrator approach but I cannot alter production tables at this time.
fruch
commented
4 years ago I think we can try the migrator approach but I cannot alter production tables at this time.
Again I never used this downgrade command, but it seems to make a copy of the tables, not changing them inplace, at least what the examples there are showing.
I guess it would be safe to test it on its own cluster/node and on production.
And the problem starts - trying to load sstable using sstableloader:
Notice that it does not even print any debug info ...
I've tried many things and then finally I decided to COPY /etc/scylla/scylla.yaml as /etc/scylla/cassandra.yaml ... and now we have debug info:
I don't understand why sstableloader is required to have "cassandra.yaml" to even start printing anything ... but ok...
Still, nothing is being inserted. The snapshot contains data from Cassandra 3.11.
Any ideas?
Thanks!
/etc/scylla/scylla.yaml: