search

scylladb / scylla-migrator

Migrate data extract using Spark to Scylla, normally from Cassandra
Apache License 2.0
54 stars 34 forks source link

Expand the tests to cover migrations from Cassandra, ScyllaDB, and Parquet #121

Closed julienrf closed 2 months ago

julienrf commented 3 months ago

To test more migration scenarios, we now spawn containers providing a Cassandra instance and second ScyllaDB instance (used as a source database in a test).

We also added new library dependencies to interact with those databases during test preparation.

As a result, running the full test suite requires significant resources. It seems to work fine on the GitHub Actions runners, but we could split things apart (e.g. first create the containers for the DynamoDB to Alternator scenario, and then for the C* to Scylla scenarios).

Fixes #114

julienrf commented 3 months ago

Waiting for #124 to remove https://github.com/scylladb/scylla-migrator/pull/121/commits/876891f52354b99822299268ebdccddde9a95699.

tarzanek commented 3 months ago

feel free to remove that commit