search

nulib / avalon

Variations-on-Video Hydra app
Apache License 2.0
3 stars 0 forks source link

Adjust S3 Sync Location #377

Closed carrickr closed 6 years ago

carrickr commented 6 years ago

New location for all materials is:

s3://preservation-cfx3wj9/avalon-masterfiles/

You'll want to recursively hit that directory as Avalon breaks up the pid, so files end up stored in the form:

s3://preservation-cfx3wj9/avalon-masterfiles/1z/40/kv/20/1z40kv20z/bunny.mp4

Also please note #374, we have some files deleted by the users that aren't on S3. So we may end up needing to sync back up to S3. So for now preserve the old syncs in a way that we can get them back please.

Toputnal commented 6 years ago

I have updated the scripts to sync S3, however, we are out of space on farchive21. On Wed, 4/25 I opened NUIT Ticket #685541 to increase the quota size on farchive21 from 8TB to 16TB. Once this happens, I will be able sync down data from the new location.

Toputnal commented 6 years ago

NUIT finished the ticket, and I have restarted the sync. Additionally, this sync will continue to run as a cron job each day.