wsidicomizer (and wsidicom) do not work with newest pydicom release 2.4

[fhnaumann]

This library (and probably wsidicom) don't work with the latest pydicom release (version 2.4; released on the 13.06.23). Everytime I use the cli command or the python interface I get a TypeError: unhashable type: 'Dataset'.

I don't know what exactly is causing the issue but so far I'm just staying on pydicom version 2.3.1 to avoid this issue.

The full error message after running the command wsidicomizer -i CMU-1.tiff -o . is

Traceback (most recent call last):
  File "/usr/local/bin/wsidicomizer", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/wsidicomizer/cli.py", line 211, in main
    cli.cli()
  File "/usr/local/lib/python3.10/site-packages/wsidicomizer/cli.py", line 154, in cli
    self.convert(
  File "/usr/local/lib/python3.10/site-packages/wsidicomizer/cli.py", line 190, in convert
    WsiDicomizer.convert(
  File "/usr/local/lib/python3.10/site-packages/wsidicomizer/wsidicomizer.py", line 219, in convert
    with cls.open(
  File "/usr/local/lib/python3.10/site-packages/wsidicomizer/wsidicomizer.py", line 141, in open
    return cls(source, label)
  File "/usr/local/lib/python3.10/site-packages/wsidicom/wsidicom.py", line 71, in __init__
    self._levels = Levels.open(source.level_instances)
  File "/usr/local/lib/python3.10/site-packages/wsidicomizer/dicomizer_source.py", line 121, in level_instances
    return [
  File "/usr/local/lib/python3.10/site-packages/wsidicomizer/dicomizer_source.py", line 122, in <listcomp>
    WsiInstance.create_instance(
  File "/usr/local/lib/python3.10/site-packages/wsidicom/instance/instance.py", line 229, in create_instance
    instance_dataset = WsiDataset.create_instance_dataset(
  File "/usr/local/lib/python3.10/site-packages/wsidicom/instance/dataset.py", line 668, in create_instance_dataset
    dataset = deepcopy(base_dataset)
  File "/usr/local/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/usr/local/lib/python3.10/site-packages/pydicom/dataset.py", line 476, in __deepcopy__
    copied.__dict__.update(copy.deepcopy(self.__dict__, memo))
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/usr/local/lib/python3.10/site-packages/pydicom/dataset.py", line 476, in __deepcopy__
    copied.__dict__.update(copy.deepcopy(self.__dict__, memo))
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/usr/local/lib/python3.10/site-packages/pydicom/sequence.py", line 86, in __deepcopy__
    copied.__dict__.update(deepcopy(self.__dict__, memo))
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 206, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/usr/local/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/usr/local/lib/python3.10/site-packages/pydicom/dataset.py", line 476, in __deepcopy__
    copied.__dict__.update(copy.deepcopy(self.__dict__, memo))
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/usr/local/lib/python3.10/site-packages/pydicom/sequence.py", line 86, in __deepcopy__
    copied.__dict__.update(deepcopy(self.__dict__, memo))
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 206, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/usr/local/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/usr/local/lib/python3.10/site-packages/pydicom/sequence.py", line 86, in __deepcopy__
    copied.__dict__.update(deepcopy(self.__dict__, memo))
  File "/usr/local/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/lib/python3.10/copy.py", line 298, in _reconstruct
    y[key] = value
TypeError: unhashable type: 'Dataset'

[erikogabrielsson]

Thanks for the report @wand555 The error seems to come from the create_sample()-method in WsiDicomizer/dataset.py. Im not sure if this error is limited to WsiDicomizer and WsiDicom. It seems that changes in pydicom 2.4.0 breaks the highdicom content dataset, and it is a know issue.

[CPBridge]

I have found the issue and submitted an issue and fix at the pydicom level:

https://github.com/pydicom/pydicom/pull/1813 and https://github.com/pydicom/pydicom/pull/1814

[erikogabrielsson]

@CPBridge fixed some of the issues, but we still get an error when doing deepcopy() on some datasets (issue).

Waiting for a fix at pydicom or that I implement a way to make the datasets without the need to deepcopy. Meantime will restrict the pydicom version to < 2.4.

[erikogabrielsson]

Fixed in 0.12.0 as we no longer need to make deepcopy of datasets.