ClimateImpactLab / downscaleCMIP6

Downscaling & bias correction of CMIP6 tasmin, tasmax, and pr for the R/CIL GDPCIR project
MIT License
136 stars 33 forks source link

Migrate storage and workflows to Oregon #640

Closed brews closed 1 year ago

brews commented 1 year ago

As R&D spins down on this project, we will try to move all the storage for this project over to the west coast (us-west1 on GCP). One of the main reasons is that this co-located the stored data with the majority of our computers.

We originally separated our compute and storage across the US west coast, the US Midwest, and Europe so that critical projects wouldn't fight each other for resource quotas. This is no longer an issue.

As part of transition, I think we would sunset and tear down the custom cluster in the midwest (defined in /infrastructure/terraform) and instead have future work run in a new namespace on workflows.cilresearch.org. This reduces the weight of maintenance.

brews commented 1 year ago

Okay, got new nodes up on cilresearch.org and migrated the downscaling workflows templates into the default namespace. They appear to run in the new cluster/region. 👍

I would like to get the permissions and templates set up so they can run in their own "downscalingcmip6" namespace without cluttering the default one. That isn't as pressing as just ensuring they run.

brews commented 1 year ago

Alright, and with this, we don't have need for a fancy, project-specific infrastructure setup, so I also removed the infrastructure/ directory in #643.

Closing.