opensafely-core / opencodelists

OpenCodelists is an open platform for creating and sharing codelists of clinical terms and drugs.
https://www.opencodelists.org
Other
31 stars 11 forks source link

Backup coding system databases #2153

Open lucyb opened 3 days ago

lucyb commented 3 days ago

As far as I'm aware, we don't currently back up our coding system databases directly. If we lost these, they might be time consuming and tricky to restore. However, we do back up the Droplet, so we can restore the droplet and get a copy of the databases that way (see #2128).

We have one database per coding system release and generally consider them to be immutable. In total they take up 38G (at the time of writing) and will get larger over time.

We only have 155GB of disk space on dokku3 and are currently using half of it. So, it's probably not a good idea to backup the databases to the same machine. We should consider backing them up to Digital Ocean Spaces Object Storage and also storing the downloaded files there.

We should manage the backups in a similar way to #2151, by using a cron job. However, since the databases aren't written to we should be able to copy the files directly rather than use the sqlite cli.

Acceptance criteria