BCDevOps / backup-container

A simple container for a simple backup strategy.
Apache License 2.0
39 stars 56 forks source link

Creating a pg_dump for specific tables #122

Open rshourou opened 7 months ago

rshourou commented 7 months ago

I want to exclude specific tables for backup, but backup.conf only accepts the list of databases that are assumed to be valid for the backup and dumps all tables in a DB. I'm wondering where I can modify pg_dump command

WadeBarnes commented 7 months ago

Can you provide more detail about your use case?

If you're looking to backup a specific schema there is some support for that. As for being selective about the tables being backed up in a given database, you'd have to customize the plugin supporting the database you're using. Plugin scripts can be found in the https://github.com/BCDevOps/backup-container/tree/master/docker folder.

I will add a word of caution. I don't think it's a good practice to be selective about what tables are being backed up from a database. This will have unexpected consequences when trying to restore data. The restore process supported by the backup container is a full drop and recreate from backup.

neophyte57 commented 7 months ago

FYI @WadeBarnes, the use case is that we have a very large table that is simply a copy of data from another system. Because it is a copy, it doesn't need to be backed up. We have this copy for convenience to run reports. Nothing references this table so referential integrity should not be an issue when restoring the backup.

WadeBarnes commented 7 months ago

When you say "very large", how large? Have you tried backing up the database using the backup container and looking at the size of resulting backup to see if the size is an issue?

You would have to customize the plugin to support this scenario. You'd have to also customize the restore functionality otherwise the table would be missing from the restored database, but perhaps that's what you're expecting.

neophyte57 commented 7 months ago

65 GB in online database @WadeBarnes image