This updates the existing backup processes, so we can run regular updates on a cronjobs again, and get them safely backed up for adjusting.
Specific details
Flags for mysql dump
We use --single-transaction to avoid locking the database, and stopping being able to add new entries to the greencheck tables. We end up wrapping the dump in a single transaction, which:
dumps the consistent state of the database at the time when START TRANSACTION was issued without blocking any applications.
We use --quick to save memory, as otherwise the dump file may exhaust the memory available to the server.
Separate dumps
We dump the files piecemeal, because when I tried dumping the entire database in one go, I kept running out of memory. When I look at the tables, like greencheck_2012 compared to regular ol' greencheck, it makes me wonder if in future we can replace them with views to save space.
As it stands it takes about an hour to export all the data and copy off the prod server.
To be handled in a separate issue
Using ansible to set up a crontab entry to run this regularly
Checking licences
I've set up FOSSA to run an automated check for licenses, and because the Ansible deployment cookbook is GPL, it was flagged up. As far as I can tell though, we are able to keep the GPL for that, and have the API codes with a separate license, so I've resolved that manually now.
This updates the existing backup processes, so we can run regular updates on a cronjobs again, and get them safely backed up for adjusting.
Specific details
Flags for mysql dump
We use
--single-transactionto avoid locking the database, and stopping being able to add new entries to the greencheck tables. We end up wrapping the dump in a single transaction, which:We use
--quickto save memory, as otherwise the dump file may exhaust the memory available to the server.Separate dumps
We dump the files piecemeal, because when I tried dumping the entire database in one go, I kept running out of memory. When I look at the tables, like
greencheck_2012compared to regular ol'greencheck, it makes me wonder if in future we can replace them with views to save space.As it stands it takes about an hour to export all the data and copy off the prod server.
To be handled in a separate issue
Checking licences
I've set up FOSSA to run an automated check for licenses, and because the Ansible deployment cookbook is GPL, it was flagged up. As far as I can tell though, we are able to keep the GPL for that, and have the API codes with a separate license, so I've resolved that manually now.