search

lobsters / lobsters-ansible

Ansible playbook for lobste.rs
ISC License
79 stars 26 forks source link

backup-cron failed to fail #36

Closed pushcx closed 11 months ago

pushcx commented 5 years ago

A week ago tarsnap emailed me a standard notice that our balance was running low and it was time to refill. I deliberately let this run below zero and tarsnap duly disabled the ability to create or list backups:

$ tarsnap --configfile tarsnap.conf --list-archives
tarsnap: Cannot read list of archive fragments: Account balance is not positive.
tarsnap: Please add more money to your tarsnap account
tarsnap: Error listing archives
tarsnap: Error exit delayed from previous errors.
$ echo $?
0

I manually ran backup-cron in prod and confirmed the prod behavior matches: the two calls to tarsnap both printed this error message.

I consider the zero exit code indicating success to be a bug in tarsnap and will file it there, but wanted to track here so I can't forget. It seems very, very unlikely but if tarsnap wontfixes the behavior we'll have to increase the complexity of the backup script (:frowning:) to detect this text (and should probably audit tarsnap for any other uncoded failures).

pushcx commented 5 years ago

I've funded the account, manually run a backup, and expect our daily backups to function normally.