DIAGNijmegen / pathology-whole-slide-data

A package for working with whole-slide data including a fast batch iterator that can be used to train deep learning models.
https://diagnijmegen.github.io/pathology-whole-slide-data/
Apache License 2.0
86 stars 24 forks source link

cuCIM backend not fully supported #57

Closed polejowska closed 4 months ago

polejowska commented 4 months ago

After changes introduced in https://github.com/DIAGNijmegen/pathology-whole-slide-data/pull/56 by @siemdejong that fixed https://github.com/DIAGNijmegen/pathology-whole-slide-data/issues/55, new errors occur therefore it seems that the cuCIM backend is not yet fully supported.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[7], line 3
      1 from wholeslidedata.iterators import create_batch_iterator
----> 3 dataLoader = create_batch_iterator(
      4     user_config=user_config, 
      5     mode='training',
      6     determinstic=True,
      7     cpus=cpus,
      8 )

File /usr/local/lib/python3.11/site-packages/wholeslidedata/iterators/batchiterator.py:159, in create_batch_iterator(user_config, mode, number_of_batches, index, update_samplers, return_info, search_paths, presets, cpus, context, determinstic, extras_shapes, buffer_dtype, iterator_class)
    156 if len(extras_shapes) > 0:
    157     buffer_shapes = buffer_shapes + extras_shapes
--> 159 dataset = build_config(config[mode])["dataset"]
    161 if number_of_batches is not None and number_of_batches > 0:
    162     batch_left = 0

File /usr/local/lib/python3.11/site-packages/dicfg/factory.py:124, in build_config(config)
    114 def build_config(config: dict):
    115     """Builds config
    116 
    117     Args:
   (...)
    121         dict: build config
    122     """
--> 124     return _ObjectFactory(deepcopy(config)).build_config()

File /usr/local/lib/python3.11/site-packages/dicfg/factory.py:26, in _ObjectFactory.build_config(self)
     25 def build_config(self):
---> 26     return self._build(self._configuration)

File /usr/local/lib/python3.11/functools.py:946, in singledispatchmethod.__get__.<locals>._method(*args, **kwargs)
    944 def _method(*args, **kwargs):
    945     method = self.dispatcher.dispatch(args[0].__class__)
--> 946     return method.__get__(obj, cls)(*args, **kwargs)

File /usr/local/lib/python3.11/site-packages/dicfg/factory.py:38, in _ObjectFactory._build_dict(self, config)
     36     config[key] = value
     37 elif _is_object(value):
---> 38     config[key] = self._build_object(value)
     39 else:
     40     config[key] = self._build(value)

File /usr/local/lib/python3.11/site-packages/dicfg/factory.py:67, in _ObjectFactory._build_object(self, value)
     65 kwargs.update(value.pop("**kwargs", {}))
     66 attribute = self._parse_object_str(object_string)
---> 67 return attribute(*args, **kwargs)

File /usr/local/lib/python3.11/site-packages/wholeslidedata/data/dataset.py:45, in WholeSlideDataSet.__init__(self, mode, associations, labels, load_images, copy_path, spacing)
     43 self._copy_path = copy_path
     44 self._associations = associations
---> 45 self._data = dict(sorted(self._open(self._associations, labels=labels).items()))
     46 self._labels = self._init_labels()
     47 self._sample_references = self._init_samples()

File /usr/local/lib/python3.11/site-packages/wholeslidedata/data/dataset.py:97, in WholeSlideDataSet._open(self, associations, labels)
     95 spacings = []
     96 for wsi_index, wsi_file in enumerate(associated_files[WholeSlideImageFile.IDENTIFIER]):
---> 97     image, spacing = self._open_image(wsi_file)
     98     data[file_key][self.__class__.IMAGES_KEY][
     99         wsi_index
    100     ] = image
    101     spacings.append(spacing)

File /usr/local/lib/python3.11/site-packages/wholeslidedata/data/dataset.py:119, in WholeSlideDataSet._open_image(self, wsi_file)
    116         raise ValueError('Load images is False, but no spacing is set')
    117     return wsi_file, self._spacing
--> 119 wsi = wsi_file.open()
    120 spacing = wsi.spacings[0]
    121 return wsi, spacing

File /usr/local/lib/python3.11/site-packages/wholeslidedata/data/files.py:34, in WholeSlideImageFile.open(self)
     33 def open(self):
---> 34     return WholeSlideImage(self.path, self._image_backend)

File /usr/local/lib/python3.11/site-packages/wholeslidedata/image/wholeslideimage.py:35, in WholeSlideImage.__init__(self, path, backend, auto_resample)
     27 """WholeSlideImage that can open en sample from whole slide images
     28 
     29 Args:
     30     path (Union[Path, str]): path to whole slide image file
     31     backend (Union[WholeSlideImageBackend, str], optional): image backend that opens and extracts regions from the whole slide image. Defaults to 'openslide'.
     32 """
     34 self._path = path
---> 35 self._backend = get_backend(backend)(path=self._path)
     36 self._auto_resample = auto_resample
     37 self._shapes = self._backend._init_shapes()

File /usr/local/lib/python3.11/site-packages/wholeslidedata/interoperability/cucim/backend.py:7, in CucimWholeSlideImageBackend.__init__(self, path)
      6 def __init__(self, path: str) -> None:
----> 7     CuImage.__init__(self, path)
      8     WholeSlideImageBackend.__init__(self, path)

TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. cucim.clara._cucim.CuImage(path: str)

Invoked with: PosixPath('/data/pathology/archives/breast/camelyon/CAMELYON16/images/tumor_062.tif')

After introducing quick fix - casting PosixPath to str in wholeslidedata/interoperability/cucim/backend.py it works

polejowska commented 4 months ago

Additional information: batch iterator is created successfully using other image backends such as ASAP, openslide with the same user_config provided (just different image_backend).