search

Ouranosinc / xclim

Library of derived climate variables, ie climate indicators, based on xarray.
https://xclim.readthedocs.io/en/stable/
Apache License 2.0
328 stars 58 forks source link

Grouper use with sbck #1204

Open redouanelg opened 2 years ago

redouanelg commented 2 years ago

Addressing a Problem?

Hello !

Many thanks for this amazing library. At the Institut Pierre-Simon Laplace (IPSL) we are many using SBCK, so seeing the xclim wrapper you developed for it was a game changer and made us leverage the power of Dask.

I am trying to make the wrapper use the Grouper, but with no success, any indication how can I proceed? I feel it has something to do with the decorators that are not used for SBCKAdjust() unlike the other sdba

Contribution

Code of Conduct

tlogan2000 commented 2 years ago

@aulemahal can you provide any guidance? This might not even be possible right now with the simple wrapper we have implemented?

aulemahal commented 2 years ago

Salut @redouanelg,

Indeed, the grouper option is not available for the SBCKAdjust subclasses. In fact the idea behind the grouping for the other adjustment methods is that you first calibrate the transfer function for each group individually, with the option of including more data than only the specific group (for example with time.dayofyear and a window). Then, the adjustment itself is done without explicit grouping, the "group" of each datapoint to correct is rather an input of the transfer function. For example, for a monthly Quantile Mapping, the quantile map has a "month" and a "quantile" dimension and when correcting a specific day we interpolate the map to get the proper factor.

What I mean is that in the code we need to loop over groups in the training phase but not in the adjust phase. The simple SBCK wrapper does the two at the same time because I have not found a generic way to access the transfer function once the SBCK object if fitted. It's been some time since I dove into the code, but I think the reason was that the data structure depend from object to object (different algorithms). It might also be that some SBCK methods are simply incompatible with the grouping idea.

Maybe the example with monthly grouping and factor interpolation is a bit to complex. It works well with Quantile Mapping, but it's not necessary to keep it. We could have restricted grouping for some methods : when interpolation is not needed, then the process can be done by looping over groups for both phases and concatenating the results afterward.

Du coup, this is work to be done. I believe the SBCK integration might need to be more granular, with a specific class wrapper for each SBCK method, instead of a single generic one. If you or any colleagues at IPSL have time to contribute, that would be wonderful! We'll be there to guide you, review and complement any PR you submit!

As for us, we have plans to explore the dOTC method, so I might find time to work on this issue in the next months!

redouanelg commented 2 years ago

Merci @aulemahal !

I see better now, I am in close contact with @yrobink the main dev of SBCK, we'll see how we can proceed and keep you in the loop if we find an optimal way. Until then we can close this issue.

aulemahal commented 2 years ago

Cool! I think I'll keep the issue open as long as this is not solved!