comeara / pillar

Pillar manages migrations for your Cassandra data stores.
https://github.com/comeara/pillar
MIT License
111 stars 64 forks source link

Add migration lock table #11

Open magro opened 10 years ago

magro commented 10 years ago

When running migrations in parallel (e.g. from integration tests running in parallel or parallel app deployments) there might occur conflicts, because the same migration is run from different clients. The error might look like this:

com.datastax.driver.core.exceptions.InvalidQueryException: Invalid
column name mycolumn because it conflicts with an existing column

To prevent this a separate migrations_lock table should/could be used, where a lock is created for the time migrations are performed (liquibase has this for example).

fkoehler commented 10 years ago

Are you sure this is still relevant as the DataStax driver waits for schema agreements?

magro commented 10 years ago

I thought that in a continous deployment scenario when multiple app servers are started simultaneously the lock table might be useful, so that only one app server instance runs migrations - to prevent race conditions.

fkoehler commented 10 years ago

This is why you wrote https://github.com/inoio/sbt-pillar-plugin/. In a continous deployment scenario one will run the migrations not in the app server but separately with the SBT task.

Of course if one does not use the plugin the requested feature would make sense but we have no user right now for it. Then pillar would need to implement this with lightweight transactions (paxos) due to the eventual consistent nature of C*

magro commented 10 years ago

Right, in our case there currently no need for it, but IMHO this would still be a useful addition - that's why I left it open. Perhaps s.o. is looking for cool things to code and stumbles over this issue :-)