search

medic / cht-sync

Data synchronization between CouchDB and PostgreSQL for the purpose of analytics.
GNU General Public License v3.0
4 stars 5 forks source link

Add culling to GH backup CI of superset dashboards #173

Open mrjones-plip opened 1 month ago

mrjones-plip commented 1 month ago

What feature do you want to improve? We offer a way to to automate backup of superset dashboards. however, the CI job we run will fill up with a backup every day. While handy - this is likely too many files which will never get used.

Describe the improvement you'd like The CI job should not only make a backup, but also cull older backups from the GH repo. Something like keeping 1/day x 7 days, 1/week x 4 weeks, 1/month x 12 months.

Describe alternatives you've considered manually cull or do nothing at all

Additional context we're hoping to have national deployments run this and a more clean list of files in backup dir will be much easier when it comes time to restore.

dianabarsan commented 1 month ago

@mrjones-plip the CI job can be setup to run on whatever schedule maintainers decide, it doesn't have to be daily. I think automatic deletions are quite dangerous. Manually cull is ok, but that should only require documentation.

mrjones-plip commented 3 weeks ago

Fair point! While think that the old adage "backup is only as good as the restore" is very applicable here, I can agree that some docs on how to cull older backups would suffice!