search

hytest-org / hytest

https://hytest-org.github.io/hytest/
22 stars 10 forks source link

Transfer data to new OSN pod buckets #396

Open amsnyder opened 8 months ago

amsnyder commented 8 months ago

The buckets that we will use on the new OSN pod have been created. Data needs to be transferred out of all the old test buckets into the new ones.

Before transferring data, other groups using the OSN pod will be contacted and the credentials for the new bucket they should work in will be shared. They should also update any workflows that read from the OSN pod to point to the new data location after the data is transferred (before it gets deleted from the test bucket).

amsnyder commented 8 months ago

@amsnyder is currently transferring conus404-hourly and @rsignell-usgs is transferring conus404-daily and conus404-monthly.

amsnyder commented 8 months ago

I've transferred all the conus404 datasets (hourly, daily_xtrm, daily, monthly) and updated the intake catalog with the new paths.

amsnyder commented 6 months ago

Plan for file transfers distributed to data contacts for review and feedback on where data is being used: https://doimspp.sharepoint.com/:x:/r/sites/gs-wma-hytest/_layouts/15/Doc.aspx?sourcedoc=%7B74388f0c-fd1c-4b84-85c8-561cf47a8163%7D&action=editnew

amsnyder commented 6 months ago

All data has been transferred and catalogs/notebooks referencing them have been updated. A few copies of old datasets remain in their old locations until the owners can update notebooks that use them.