ClimateImpactLab / dodola

Containerized application for running individual tasks in a larger, orchestrated CMIP6 bias-adjustment and downscaling workflow.
https://climateimpactlab.github.io/dodola/
Apache License 2.0
16 stars 8 forks source link

Fix wetday frequency correction #174

Closed emileten closed 2 years ago

emileten commented 2 years ago

essential changes :

secondary changes :

  1. in the core function, directly operate on the data array object within the dataset -- which means I have to introduce a var parameter -- default value is 'pr', but in tests we have other variable names. Need that parameter in the service as well.
emileten commented 2 years ago

@dgergel I requested your review for the method check.

@brews, in the computational side. I tested this on both Jupyter and Argo, and this ran within 3 mins using around 15GB of memory (that's because of the huge numpy sample array that I have to create, which has the shape of the GCM data). That fits in ~20GB resources for that step. We might need to increase slightly this number so that the ERA-5 data fits in.

But it will fit in the node and run fast.

emileten commented 2 years ago

Thanks for cleaning this up, @emileten.

My one suggestion is to rename this new var argument to variable. This makes it consistent with the other functions and methods that grab a variable name. ...I see "var" and I think "variance" but that's an aside....

Thanks @brews, I changed this !