search

ClimateImpactLab / downscaleCMIP6

Downscaling & bias correction of CMIP6 tasmin, tasmax, and pr for the R/CIL GDPCIR project
MIT License
128 stars 31 forks source link

Move workflows from midwest to west coast storage #639

Closed brews closed 1 year ago

brews commented 1 year ago

Update Argo Workflows so they use the new storage on the west coast (from PR #638) rather than the older storage in the midwest.

Related to #640

brews commented 1 year ago

If this merges as it is right now, it would likely run but the cost from data transfer fees would be extremely high because the machines (in Iowa) and storage (in Oregon) are not co-located.

I'm thinking the goal here is to get these workflows deployed and running on workflows.cilresearch.org (which is mostly in Oregon) and then we sunset the specialized cluster we were running for this project. I need to change a few things and read up on a few details for workflows.cilresearch.org before I can actually implement this. I don't see why I can't have it done before the end of the week though.

This is something that will likely take more than one PR and I should write up some issues to outline the work and explain what's going on.

Edit: Outlining thinking in #640.