search

scylladb / scylla-migrator

Migrate data extract using Spark to Scylla, normally from Cassandra/parquet files. Alt. from DynamoDB to Scylla Alternator.
https://migrator.docs.scylladb.com/stable/
Apache License 2.0
55 stars 34 forks source link

Fix a bug with migrating null regular columns #16

Closed goakley closed 3 years ago

goakley commented 4 years ago

If the migrator encounters a row containing entirely null values in the regular columns, it skips over that row, whereas it should insert the primary key values. This is due to the logic in explodeRow skipping all rows with a null writetime (which null-valued columns have).

An example row that is affected by the bug:

CREATE TABLE test (key int PRIMARY KEY, value int);
INSERT INTO TEST (key) VALUES (1);

The new logic allows inserting rows with writetime marked as unset.

tarzanek commented 3 years ago

thank you @goakley ! I rebased and merged it as https://github.com/scylladb/scylla-migrator/pull/42 I just realized I only included the reference to here, is that a problem? (all credit goes to you, but it's referenced and not explicitely mentioned in commit message, sorry for that, if you will insist I will rewrite history, let me know in such case (either here or on support/users scylla slack (Lubos Kosco) )