sky-uk / cqlmigrate

Cassandra schema migration library
BSD 3-Clause "New" or "Revised" License
47 stars 29 forks source link

CQL migrate fails due to race condition of updating schema metadata after acquiring lock #49

Open chrishughes25 opened 7 years ago

chrishughes25 commented 7 years ago

When deploying to 2 nodes, the first gets the lock, runs migrate successfully, happy days. The second then got the acquired the lock, skips the bootstrap but doesn't seem to have the updated schema metadata so it tries to apply the "create schema updates table". This fails with an exception as it already exists. [2017-01-06 11:58:09,563] lcm INFO [main] u.s.c.Lock - [] Attempting to acquire lock for 'channel_metadata.schema_migration', using client id '93155931-75b4-4fae-8be9-3dc51c111f4c' [2017-01-06 11:58:09,584] lcm INFO [main] u.s.c.CassandraLockingMechanism - [] Lock currently held by Row[false, channel_metadata.schema_migration, 948f2e5f-8447-4093-bd7f-cf32be7289f1] [2017-01-06 11:58:10,095] lcm INFO [main] u.s.c.CassandraLockingMechanism - [] Lock currently held by Row[false, channel_metadata.schema_migration, 948f2e5f-8447-4093-bd7f-cf32be7289f1] [2017-01-06 11:58:10,644] lcm INFO [main] u.s.c.CassandraLockingMechanism - [] Lock currently held by Row[false, channel_metadata.schema_migration, 948f2e5f-8447-4093-bd7f-cf32be7289f1] [2017-01-06 11:58:11,171] lcm INFO [main] u.s.c.CqlMigratorImpl - [] Loading cql files from [/tmp/cqlmigrate7688832900537474535] [2017-01-06 11:58:11,180] lcm INFO [main] u.s.c.KeyspaceBootstrapper - [] Keyspace found, not applying bootstrap.cql [2017-01-06 11:58:11,206] lcm ERROR [main] u.s.c.CqlLoader - [] Failed to execute cql statements [CREATE TABLE schema_updates (filename text primary key, checksum text, applied_on timestamp);]: Table channel_metadata.schema_updates already exists

chrishughes25 commented 7 years ago

possible fix: use CREATE IF EXISTS, and remove metadata check.

jsravn commented 6 years ago

Thanks @chrishughes25 for finding this. Do you mind submitting a PR?