search

k0lter / autopostgresqlbackup

Automated tool to make periodic backups of PostgreSQL databases
52 stars 17 forks source link

Fails to detect when DB dumps fail, and saves useless backups #2

Closed NightTsarina closed 10 months ago

NightTsarina commented 4 years ago

Note: this bug was also reported in Debian: #956083

Due to a human error, today I had to resort to backups to recover data from a PostgreSQL database. In the worst possible moment, I realised that ALL of the backups we had of this database are useless.

Turns out that when dumping the database to a huge text file, the system was running out of disk space, and so the file is truncated.

Instead of detecting the problem, aborting the backup, and sending an email to the administrator, autopostgresqlbackup assumes the dump finished well, proceeded to compress the truncated dump, and continued as if nothing had happened. This had been going on for months without me realising, and so I have no useful backup.

I believe this is critical, because this bug has made me lose real world data today.

The sad part is that the bug is pretty trivial to fix: all commands should be checked for non-zero return codes, probably by setting the '-e' flag at the beginning of the script.

k0lter commented 1 year ago

@NightTsarina With the last commits (the whole script was almost rewritten), the error handling was improved. In such case, despite the lack of disk space on backup storage, the mail will be sent to the administrator.

k0lter commented 10 months ago

I'm closing this bug as it is already solved.