Backup script fails silently

mpaik commented 5 years ago

The script src/scripts/backup doesn't check whether the database dump from pgsql completes successfully, as I now know to my cost. Looking over the resulting database.sql (after trying, and failing, to upgrade), I find this:

pg_dump: [archiver (db)] query failed: FATAL:  terminating connection due to administrator command
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
pg_dump: [archiver (db)] query was: SELECT f.tableoid, f.oid, f.fdwname, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid = f.fdwowner) AS rolname, f.fdwhandler::pg_catalog.regproc, f.fdwvalidator::pg_catalog.regproc, (SELECT pg_catalog.array_agg(acl) FROM (SELECT pg_catalog.unnest(coalesce(f.fdwacl,pg_catalog.acldefault('F',f.fdwowner))) AS acl EXCEPT SELECT pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('F',f.fdwowner)))) as foo) AS fdwacl, (SELECT pg_catalog.array_agg(acl) FROM (SELECT pg_catalog.unnest(coalesce(pip.initprivs,pg_catalog.acldefault('F',f.fdwowner))) AS acl EXCEPT SELECT pg_catalog.unnest(coalesce(f.fdwacl,pg_catalog.acldefault('F',f.fdwowner)))) as foo) AS rfdwacl, NULL AS initfdwacl, NULL AS initrfdwacl, array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value) FROM pg_options_to_table(f.fdwoptions) ORDER BY option_name), E',
    ') AS fdwoptions FROM pg_foreign_data_wrapper f LEFT JOIN pg_init_privs pip ON (f.oid = pip.objoid AND pip.classoid = 'pg_foreign_data_wrapper'::regclass AND pip.objsubid = 0) 
pg_dumpall: pg_dump failed on database "gitlab", exiting

This file is, of course, tgz'ed up, which means that there's no way to check whether it completed successfully without digging into the archive, and nothing is reported on the CLI:

Password:
stopping GitLab
backup config
compress gitlab folder
backup gitlab database
compress everything to a backup file
start GitLab
backup completed!

Not great.

jboxberger commented 5 years ago

Yeah, you're right. I am working on a solution. I will rework and re-think the whole process.

Until then i would reccomend you to use the native gitlab backup routine. The Backups are stored in /volume1/docker/gitlab/backups

Backup: sudo /usr/local/bin/docker exec -it synology_gitlab bash -c "sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1" Restore: /usr/local/bin/docker exec -it synology_gitlab bash -c "sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production BACKUP=<timestamp_of_backup>"

Hope this helps.

Kind Regards

jboxberger commented 5 years ago

Hello mpaik,

thanks to your issue i have removed the custom backup solution and recommend from now to use the default gitlab bacup/restore functions here. Thank you for the bug report.

Kind Regards.

jboxberger / synology-gitlab-jboxberger

Backup script fails silently #10