Glacier redundancy - Githubissues

cedardevs / onestop-clients

Client applications and libraries for interacting with the OneStop search API and event stream

1 stars 0 forks source link

Glacier redundancy #16

Open dneufeldcu opened 3 years ago

dneufeldcu commented 3 years ago

Summary

As an	archivist
I want to be able to	ensure my data is sent to a long-term archive is a separate region (us-west-2)
So that I can	that I have a backup copy in a different geography that supports COOP

Tasks

[x] Create vault
[x] Upgrade onestop-python-client to support sending data to glacier in a separate region
[x] Modify code to use s3 api over existing use of glacier
[x] Create utility class to restore from glacier for testing purposes

JeffreyHuynh1 commented 3 years ago

Requires 3-5 hours for archival retrieval after initiating a job

JeffreyHuynh1 commented 3 years ago

In order to retrieve an item in S3 glacier using the S3 client you need to restore the object from glacier and then use get_object

JeffreyHuynh1 commented 3 years ago

Created various scripts in archive_client_integration to test s3 to glacier upload, glacier restore to s3, etc. Might be good create different files for each script for better organization/management.