ArtResearch / artresearch.net

ArtResearch ResearchSpace application hosted at https://artresearch.net
5 stars 1 forks source link

daily backup for all data to S3 #432

Closed lklic closed 1 year ago

lklic commented 2 years ago

I think we should set up backup tasks for the following data:

If there is anything else I am missing please add it to the issue here.

We can use this docker image as a starting point to manage the backups: https://github.com/istepanov/docker-backup-to-s3

I think we should modify the dockerfile to compress the folder of the blazegraph journal.

Also, we should enable versioning for the S3 bucket so that we upload a daily backup that replaces the old file, but keep older versions for 60 days. Please see this ref for managing versioned files in S3: https://docs.aws.amazon.com/AmazonS3/latest/userguide/manage-objects-versioned-bucket.html