search

scylladb / scylla-migrator

Migrate data extract using Spark to Scylla, normally from Cassandra/parquet files. Alt. from DynamoDB to Scylla Alternator.
https://migrator.docs.scylladb.com/stable/
Apache License 2.0
55 stars 34 forks source link

Strip trailing zeros throws error #35

Open fcosta-td opened 3 years ago

fcosta-td commented 3 years ago

I am unable to run migrator with this option, true and false, due to the following error:

20/11/02 11:45:34 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, spark-master, 44951, None)
20/11/02 11:45:34 INFO BlockManagerMasterEndpoint: Registering block manager spark-master:44951 with 366.3 MB RAM, BlockManagerId(driver, spark-master, 44951, None)
20/11/02 11:45:34 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, spark-master, 44951, None)
20/11/02 11:45:34 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, spark-master, 44951, None)
20/11/02 11:45:34 INFO StandaloneSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
Exception in thread "main" DecodingFailure(Attempt to decode value on failed cursor, List(DownField(stripTrailingZerosForDecimals), DownField(target)))
fcosta-td commented 3 years ago

Additionally, it would be great if you could create releases every time you merge changes into master.

tarzanek commented 3 years ago

The weird error " Exception in thread "main" DecodingFailure(Attempt to decode value on failed cursor, List(DownField(stripTrailingZerosForDecimals), DownField(target))) " just says it cannot parse the config file properly

tarzanek commented 3 years ago

can you add your config? or move it to proper section?

tarzanek commented 3 years ago

if you notice the option it's part of "target" section:

https://github.com/scylladb/scylla-migrator/blob/master/config.yaml.example#L98

so yaml is sensitive about white space you need to make it properly white space alligned

(and I know the error sucks, I'll try to figure out if the error can be more human friendly)

@fcosta-td