golang-migrate / migrate

Database migrations. CLI and Golang library.
Other
15.14k stars 1.39k forks source link

feat: an option for crdb to wait util the lock is released to mimic pg locks behaviour #1076

Open vgarvardt opened 5 months ago

vgarvardt commented 5 months ago

In PostgreSQL pg_advisory_lock() is used to put a DB-level lock before applying migrations. The side-effect of this way of locking is that operation is waiting indefinitely until the lock can be acquired.

CockroachDB is pg-compatible DB that does not support advisory locks, so locking is implemented using the lock table. The main issue when trying to switch from PG to CRDB is that migration is failing right away if another process is already applying them to the DB. For most of the cases this is fine, but for some, e.g. mine, some apps are already relying on the side-effect of advisory locks and apps are starting to conflict with each other and failing, instead of waiting and running sequentially.

This PR adds retries feature to the CRDB that mimics pg lock behaviour, so that apps may continue to use migration tool capabilities to apply migrations sequentially instead of managing this behaviour in the calling side. Implementation is aligned with the retries/backoff in yugabytedb, but the default value for the retry is set to 0 to keep the current behaviour unchanged.

PS: my editor applied some formatting to the edited files that are not directly related to the change, pls let me know if I need to rollback some of them, like imports sorting

coveralls commented 5 months ago

Coverage Status

coverage: 56.591% (+0.3%) from 56.335% when pulling f4d81f95d11ca2b9ecdd45171911efb61f290431 on vgarvardt:feat/crdb-lock-wait into 0c456c49a6debb0b2deb5a97a10d9e85d157b9d5 on golang-migrate:master.