whisthq / whist

Whist Browser
https://whist.com
Apache License 2.0
7 stars 3 forks source link

Make DB Migrations atomic #1510

Closed jmathur25 closed 3 years ago

jmathur25 commented 3 years ago

Bug Description See https://tryfractal.slack.com/archives/C013GT6T86R/p1617214816077000. Summary:

Right now when a webserver PR merges, two workflows trigger:

The former redeploys the webserver with the new code, the latter applies new schema at main-webserver/db_migration/schema.sql to the live db. This leads to the following possibility:

  1. webserver is doing a db operation
  2. the schema gets changed, but the corresponding webserver ORM is not updated
  3. the operation in 1) errors out because the ORM does not match the schema
  4. the webserver redeploys, and now has an in-sycn ORM

A manifestation of this issue (the user_containers.lock column was deleted):

Screen Shot 2021-05-10 at 11 49 27 AM

We fix this by tying our webserver redeploys to db migrations. The migration must be performed after the webserver is (briefly) taken down and before it is redeployed with the new code (including new ORM).

philippemnoel commented 3 years ago

context:

Capture d’écran, le 2021-04-15 à 23 04 11
philippemnoel commented 3 years ago
Capture d’écran, le 2021-04-15 à 23 04 35