Cleanup odl-operations-backups s3 bucket

[Ardiea]

Description/Context

mike So I was looking at vantage this morning re: Carey’s question for us in the checkin and I found an s3 bucket odl-operations-backups which I think needs to go / get cleaned up substantially. 54TB in 600,000 objects, unchanged in 15 months.

Plan/Design

This bucket mostly contains old backups of various applications / databases / datastores that are so old they are no longer relevant or useful. Most of what is found in this bucket can probably be removed but use your discretion.

[Ardiea]

Proposed to delete:

• consul-operations/* Consul snapshots, most recent of which is Dec 2021, oldest is Feb 2017
• consul/* Just one snapshot from Feb 2017, probably testing from above.
• mitx-rp-cypress-archives/* Unknown DB dumps from 2016
• mitx-rp-dogwood-archive/* edx tracking log backups from 2017
• mongodb-mitx-production/* Encrypted mongodb transaction logs / backups from May 2021.
• mongodb-mitx-rp/* Encrypted mongodb transaction logs / backups from April 2017
• mysql-draft-mitx-rp/* Encrypted mysql transaction logs / backups from Jan 2017
• mysql-live-mitx-rp/* Encrypted mysql transaction logs / backups from Jan 2017
• mysql-mitx-production-residential-draft/* Encrypted mysql transaction logs / backups from Dec 2021
• mysql-mitx-production-residential-live/* Encrypted mysql transaction logs / backups from Dec 2021
• mysql-mitx-production-xqwatcher/* Encrypted mysql transaction logs / backups from Dec 2021
• mysqldump-mitx-production-residential-draft/* Encrypted mysql backups from August 2017
• mysqldump-mitx-production-residential-live/* Encrypted mysql backups from August 2017
• prod_repos/* Encrypted ??? Backups from Dec 2021
• repos/* Encrypted ??? Backups from Dec 20211
• vault-production-snapshot-* Vault snapshots in the bucket root, newest of which is from Jan 2022.

This bucket contains nearly 600,000 objects in 52.68TB of data costing approx $1150.0 / m or approx 2.25% of our monthly AWS spend.

[blarghmatey]

This all looks good. Anything that looks like tracking logs we will want to keep, but the DB backups and the repos/prod_repos can get purged.

[Ardiea]

Verified storage usage is down to ~3 GB. Setup a manual intelligent archiving life cycle rule that Vantage suggested that will eventually cut costs on the remaining items in this bucket by a further 90% down to just $3-4/month.