Open dgergel opened 4 years ago
@dgergel - Would you mind sharing your prototype concept? The BCSD disaggregator I was thinking of in the roadmap was a temporal disaggregator BUT I'm sure there are some interesting spatial methods to consider here as well. I think the criteria I'd put on accepting something like this fall as below:
@jhamman a question from the criteria you listed: would xESMF
fall under the category of adding too many dependencies, or would you be open to it being used?
The spatial disaggregator I have in mind would be implementing the NASA-NEX daily method, which applied bias correction at the native model resolution and then spatial disagg to move from the native model resolution to the finer obs resolution. The rough steps are:
I've been thinking about how to adapt my prototype to use a similar fit/predict API. What I roughly have in mind is the fit
step would be computing the scaling factor and bilinearly interpolating it, and then the predict
step would be applying the scaling factor to the obs climo. Does that sound reasonable to you?
I'd like to avoid a dependency on xESMF (esmf really). For the workflow you've highlighted here, I think we can think of the a set of recipes that implement end-to-end downscaling patterns (like NASA-NEX). I don't think there is any problem adding xESMF to the doc build but I think we can stick to the principle that regridding is out of scope as built-in functionality in scikit-downscale. Ideally, all the regridding stuff can be made plug-and-play such that arbitrary workflows can be composed using a combination of tools (xarray/scikit-downscale/xesmf/etc.).
@jhamman fair enough. By adding xESMF to the doc build, do you mean a recipe for using xESMF as an intermediate step external to the workflow, or something beyond that?
Two options I'm thinking of that would accomplish this:
fit
step, use xESMF outside of the fit/predict API to interpolate the scaling factor, and include a "recipe" in the documentation for how to do this, and then apply the scaling factor as the predict
stepfit
step would include one or more utility functions that draw on scipy to accomplish regridding solely for the scaling factor, but this wouldn't be built-in functionality that we use in other parts of scikit-downscale. I already coded this up in my prototype actually by taking out some xESMF source code. @jhamman fair enough. By adding xESMF to the doc build, do you mean a recipe for using xESMF as an intermediate step external to the workflow, or something beyond that?
Yes. I'm just thinking a recipe (notebook or script of some kind) that we can include in the documentation showing how to use scikit-downscale with xesmf. (your options 1).
I'm having a bit of a hard time visualizing your second option. Can you provide an example (pseudocode or notebook if available)?
@jhamman what I had meant in my second option was that I was directly using source code from xesmf to apply regridding weights on dask workers, as I'd had trouble getting xesmf to work with dask distributed. Since then I've gotten it up and running, so I think a recipe in the docs would work well. I'm in the process of updating my prototype notebook and also adapting it to the fit
/predict
API so I'll share it once it's a bit more cleaned up.
Circling back on this - what I have in mind is a recipe like this:
fit
a scaling factor **predict
by adding or multiplying the scaling factor to the obs-res daily climatology **scikit-downscale
scikit-downscale
The steps in scikit-downscale would call a SpatialDisaggregator
class that has fit
and predict
methods. I have an initial implementation you can check out here.
Let me know your thoughts on this @jhamman - if this all sounds reasonable, I'll put together a PR.
@jhamman (and any others who might want to weigh in on this) I was wondering what your plan is for adding functionality for spatial disaggregation? Are you still thinking about something along the lines of
bcsd_disaggregator
as mentioned in the readthedocs? I have a prototype version of spatial disagg that I put together a few months ago that I could put into a PR in the next few weeks. Would be great to hear what you already have in mind for this piece so that I can make sure that my prototype would be along those lines.