New Automated Backup Script

[preaction]

The current backup script, to save time, only backs up the last few million reports. These are there reports that are most likely to change. However, updating which millions of reports to backup is a manual process.

We need a program to run to backup from a database. On certain tables in that database, we need to be able to only back up the last N rows (thousand, million, etc), and we need those backup files to contain a certain number of rows. This should probably be configurable in a file, and the backup program then run from a cron job.

So, the files that this thing produces should likely look like this:

# 10M - 11M reports in "cpantesters.metabase" table
cpantesters-metabase-10M-20160101.sql.gz
# 11M - 12M reports in "cpantesters.metabase" table
cpantesters-metabase-11M-20160101.sql.gz
# 12M + reports (since we only have 12.4 million in this example)
cpantesters-metabase-12M-20160101.sql.gz

# The entire "cpantesters.statistics" table on 2016-01-01
cpantesters-statistics-20160101.sql.gz

If a file cannot be written due to disk space issues, the entire backup should be deleted and a nastygram printed to STDERR, as it is invalid.

[preaction]

The backup program should only keep a given number of full backups available. This was the main problem we were having with the current backup.