search

patka / cassandra-migration

Schema migration library for Cassandra
MIT License
152 stars 47 forks source link

Add leader election to avoid schema disagreements with concurrent migrations #43

Closed adejanovski closed 5 years ago

adejanovski commented 5 years ago

When starting multiple distributed processes at the same time, migrations will be executed at the same time, often resulting in schema conflicts. This commit adds leader election through LWTs so that only a single process will perform the migration at once on a given keyspace. Leader election will be optional and deactivated by default. IF NOT EXISTS clauses were added to the CREATE TABLE statements in order to avoid conflicts there as well. Both changes are disabled if the Cassandra version is below 2.0 since we would miss both LWTs and IF NOT EXISTS features.

We're using this lib in Reaper and we're moving towards more concurrency with the introduction of the sidecar mode, which creates more risks of getting schema disagreements without proper consensus. We've implemented it in our latest master but feel that this mechanism should be provided by the migration lib.

I made it so that the default remains the same, and automatically disabled features that didn't exist in 1.2 in case you want to keep supporting old versions of Cassandra.

patka commented 5 years ago

Hi,

thanks for taking the time to build this feature and I agree that this is the place to put it rather than a software using this library. I will have a closer look on the changes in the next couple of days and get back to you.

Cheers Patrick

adejanovski commented 5 years ago

Hi @patka, I've pushed another commit with the requested changes. Let me know what you think.

adejanovski commented 5 years ago

Hi @patka , no worries at all and thanks a lot for merging this PR. Could you, by any chance, cut a new release with that feature in? It would allow us to remove the consensus code from Reaper and rely on this new version.

patka commented 5 years ago

Sure, that was my plan all along as I can easily imagine you do not want to build from master yourself :-) I see that I find some time for that on the weekend.

adejanovski commented 5 years ago

Awesome, thanks again!

patka commented 5 years ago

I tried to release it on the weekend but the build fails on my local machine because of the concurrency test. Interestingly it does not fail on the TeamCity instance. I will have a closer look what the problem is but rest assured, I am on it.

adejanovski commented 4 years ago

Sorry you're having trouble releasing this @patka . Is there anything I can help you with?

patka commented 4 years ago

All fine. I used a bigger machine :) I just released it. Give it a few hours to be distributed on the maven repo mirrors.

adejanovski commented 4 years ago

Awesome, thanks a lot!