Fix working with latest dask

DigitalSlideArchive / HistomicsTK

A Python toolkit for pathology image analysis algorithms.

https://digitalslidearchive.github.io/HistomicsTK/

Apache License 2.0

395 stars 117 forks source link

Fix working with latest dask #1145

Open manthey opened 1 week ago

manthey commented 1 week ago

See PR #1144; dask introduced a breaking change.

Superficially, my guess is that it fails to unpickle tiles correctly.

manthey commented 1 week ago

Specifically, this changes how the tiles are serialized and deserialized in dask. A tile (large_image.tilesource.tiledict.LazyTileDict) is a subclassed dictionary. In dask<2024.11 this serialized and deserialzes as a LazyTileDict. In recent dask, it deserializes as a dict.

manthey commented 1 week ago

There is now an issue in distributed for this problem: https://github.com/dask/distributed/issues/8932