The cli tiling pipeline no longer worked after the recent changes in dlup==0.4.0 and ahcore. This has been fixed in this PR.
setup.py
Changed dlup version in setup.py to dlup>=0.4.0 (because code is not supported by older dlup version due to different imports and return types)
tiling.py
Make work with dlup 0.4.0 and use PYVIPS backend and "vips" as internal handler
Pass dataset grid to writer, otherwise the grid will be initialized from the coordinates of the first sample which is problematic when using a mask; the first tile sampled from the dataset is not necessarily sampled from the top left corner of the dataset grid when using a mask
Transpose array in def _generator such that channels are in the first dimension (the writer expects this)
Provide crop and mask_threshold as extra_metadata to the writer (this metadata is expected by the database model CacheDescription, but currently missing in the written files)
Do not apply the color profile to the thumbnail unless it was also applied to the saved tiles. This way, the colors in the thumbnail are more representative of what the actual saved data looks like
Put the args argument first in def wrapper and explicitly declare arguments in partial, otherwise the wrapper does not get correctly initialized in pool.imap_unordered(partial_wrapper, images_list) (it will always try to pass images_list as the first function argument)
writers.py
In Writer.init_writer the color_profile data was wrapped in a tuple. Was there a good reason for this? I removed it and seems to work fine for both h5 and zarr (before with tuple it would crash when using zarr writer)
readers.py
Using the tiling pipeline without color profiles applied, it will by default always write the color_profile of the slide image (if present) as metadata to the h5/zarr file. In FileImageReader, a NotImplementedError is raised if self._metadata["has_color_profile”], because color profiles are not yet supported in the readers. Because of this the reader could not open the files written by the tiling pipeline. I changed this to logging a warning instead of raising an error, because I don't think the reader is expected to always apply color profiles if they are available; this should be optional. Therefore I think a warning is sufficient.
The cli tiling pipeline no longer worked after the recent changes in dlup==0.4.0 and ahcore. This has been fixed in this PR.
setup.py
setup.pytodlup>=0.4.0(because code is not supported by older dlup version due to different imports and return types)tiling.py
dlup 0.4.0and usePYVIPSbackend and"vips"as internal handlerdef _generatorsuch that channels are in the first dimension (the writer expects this)cropandmask_thresholdasextra_metadatato the writer (this metadata is expected by the database modelCacheDescription, but currently missing in the written files)argsargument first indef wrapperand explicitly declare arguments inpartial, otherwise the wrapper does not get correctly initialized inpool.imap_unordered(partial_wrapper, images_list)(it will always try to passimages_listas the first function argument)writers.py
Writer.init_writerthe color_profile data was wrapped in a tuple. Was there a good reason for this? I removed it and seems to work fine for both h5 and zarr (before with tuple it would crash when using zarr writer)readers.py
FileImageReader, aNotImplementedErroris raised ifself._metadata["has_color_profile”], because color profiles are not yet supported in the readers. Because of this the reader could not open the files written by the tiling pipeline. I changed this to logging a warning instead of raising an error, because I don't think the reader is expected to always apply color profiles if they are available; this should be optional. Therefore I think a warning is sufficient.