search

caciviclab / disclosure-backend

DEPRECATED (We're working on the `disclosure-backend-static` repo instead) A Django app to download, extract and load campaign finance and lobbying activity data from the California Secretary of State's CAL-ACCESS database as well as local Netfile jurisdictions
https://github.com/caciviclab/disclosure-backend-static/
11 stars 20 forks source link

Automated database backups #41

Open adborden opened 9 years ago

adborden commented 9 years ago

We're adding more stuff to our database which is not easy to fetch/automate, so it's important we have backups.

adborden commented 9 years ago

@seanius said he could cover this.

adborden commented 8 years ago

We should also be backing up MEDIA_ROOT because we'll be hosting candidate photos there, as well as settings_local.py. Feel free to split these into alternative tickets.

tdooner commented 8 years ago

If you want to take this on @seanius, fine, but I think it's better to err towards making our database state recoverable via some import process rather than treat the data in it as sacred. Anyway, this strikes me as pretty low priority.

adborden commented 8 years ago

We have folks adding curated data to admin. I'd hate to lose it because I typo'd a chef recipe.

The recoverable part is important too, but in the event of a disaster, I just want to make sure we aren't having folks re-do their hard work.

bcipolli commented 8 years ago

As mentioned in #14, we should have a command to make these manual annotations into fixtures and commit them to some repo.

I agree with @adborden that it should be a relatively high priority--we could lose those data, and those fixtures are necessary for setting up test systems.

I agree with @tdooner that data pulled from automated sources should not be backed up.

bcipolli commented 8 years ago

We can simply schedule a cron job for #202 for the RC1. We don't need a full database backup, just a backup of the manual changes made.

adborden commented 8 years ago

I have this running on my own machine via 6 hour cron:

$ ssh opencal.opendisclosure.io mysqldump --user root -p$DB_PASSWORD opendisclosure \| gzip > opendisclosure-`date +%FT%T%z`.sql.gz