Ouranosinc / xclim

Library of derived climate variables, ie climate indicators, based on xarray.
https://xclim.readthedocs.io/en/stable/
Apache License 2.0
333 stars 59 forks source link

Fix wrong unpickling with dask 2024.11 #1993

Closed aulemahal closed 2 weeks ago

aulemahal commented 2 weeks ago

Pull Request Checklist:

What kind of change does this PR introduce?

The issue introduced with dask 2024.11 concerned the passing of arguments through the dask graph. Most SDBA classes are children of Parametrizable, itself a child of dict. When stored in the graph, objects like this are pickled. and then repickled when the function is actually executed.

For some reason, the repickling stopped working and objects where reinstated as dictionaries instead of their subclasses. This particularly touched the Grouper, one of the only such objects actually been passed through dask-backed computations.

Inheriting from dict was once not recommended but I think this is not the case anymore. Anyway, I found that inheriting from collections.UserDict actually solved the issue (after adapting the internals). Not sure if I should raise the issue back to dask...

Does this PR introduce a breaking change?

I changed how all classes of xclim.sdba are made, but in theory the change is at a low, private level.

Other information:

coveralls commented 2 weeks ago

Coverage Status

coverage: 89.398%. remained the same when pulling 58f0fb397302791fa4cf8009fbf3df51d6ac59d5 on fix-dask-2024-11 into e0a0c88f9854a18236dc9af7505d2cd10883351f on main.