scylladb / scylla-migrator

Migrate data extract using Spark to Scylla, normally from Cassandra/parquet files. Alt. from DynamoDB to Scylla Alternator.
https://migrator.docs.scylladb.com/stable/
Apache License 2.0
55 stars 34 forks source link

Validate TTL if it gets moved properly #59

Open tarzanek opened 2 years ago

tarzanek commented 2 years ago

Per https://docs.scylladb.com/getting-started/time-to-live/ we might need to decrease TTL written so we don't reset it, when taken with TTL function from source DF

https://docs.scylladb.com/kb/ttl-facts/

tarzanek commented 2 years ago

TTL function gets current TTL for the data, so source wise this looks OK

tarzanek commented 2 years ago

Does it need adjusting for PK ? PK TTL should pick the biggest out of all TTLs

tarzanek commented 2 years ago

also does it need adjusting for default, if the default TTL varies between both tables?

tarzanek commented 2 years ago

validate different columns TTL (so k, a,b,c and use 3 inserts to insert a,b,c with different TTLs) and validate target also has proper TTLs (so 3 different for each column)