search

wellcomecollection / goobi-infrastructure

Wellcome Collection digital workflow infrastructure
MIT License
0 stars 3 forks source link

Add an S3 lifecycle rule to Standard-IA for objects in the working storage bucket #423

Closed alexwlchan closed 2 years ago

alexwlchan commented 2 years ago

Currently everything stored in s3://wellcomedigitalworkflow-workflow-data is stored in the S3 Standard class, even though the vast majority of it is being accessed very infrequently.

We should set up a lifecycle rule to cycle objects to Standard-IA – this would knock a decent chunk off the cost of the bucket with minimal work required on our part.

Here's an example of how to configure this in Terraform: https://github.com/wellcomecollection/storage-service/blob/main/terraform/modules/critical/s3_replica_primary.tf#L9-L17

mgeerdsen commented 2 years ago

This has been applied to stage earlier this week and most of the data has been migrated to standard_ia. As that was successful, we have now applied the change to production.