search

Project-MONAI / tutorials

MONAI Tutorials
https://monai.io/started.html
Apache License 2.0
1.83k stars 678 forks source link

Detection tutorial does not read original Luna files #1102

Closed taborzbislaw closed 1 year ago

taborzbislaw commented 1 year ago

I am trying to run detection tutorial (https://github.com/Project-MONAI/tutorials/tree/main/detection) using original LUNA16 files in mhd+raw format

Environment is as follows:

MONAI version: 1.0.1 Numpy version: 1.23.5 Pytorch version: 1.13.0 MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False MONAI rev id: 8271a193229fe4437026185e218d5b06f7c8ce69

Optional dependencies: Pytorch Ignite version: 0.4.10 Nibabel version: 4.0.2 scikit-image version: 0.19.3 Pillow version: 9.3.0 Tensorboard version: 2.11.0 gdown version: 4.6.0 TorchVision version: 0.14.0 tqdm version: 4.64.1 lmdb version: 1.4.0 psutil version: 5.9.4 pandas version: 1.5.2 einops version: 0.6.0 transformers version: 4.21.3 mlflow version: 2.0.1 pynrrd version: 1.0.0

According to the tutorial, I should run: python3 luna16_prepare_env_files.py python3 luna16_prepare_images.py -c ./config/config_train_luna16_16g.jsonn

Script luna16_prepare_env_files.py runts correctly. When I run luna16_prepare_images.py I got an error when trying to read mhd+raw files. The mhd+raw files are correct (I can read them in external software). The error output is attached below. What additional requirements should be installed to correctly read mhd+raw files?

The next issue is that at google drive at https://drive.google.com/drive/folders/1JozrufA1VIZWJIc5A1EMV3J4CNCYovKK you have shared resampled nifti files. But there are only about 600 files at google drive while in LUNA16 there are more than 880 files.

UserWarning: required package for reader itkreader is not installed, or the version doesn't match requirement. warnings.warn(

2022-12-12 14:07:16,152 - NibabelReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.250397690690072950000431855143.mhd',).

2022-12-12 14:07:16,153 - PILReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.250397690690072950000431855143.mhd',).

2022-12-12 14:07:16,153 - NumpyReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.250397690690072950000431855143.mhd',).

2022-12-12 14:07:16,157 - NrrdReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.250397690690072950000431855143.mhd',).

2022-12-12 14:07:16,160 - PydicomReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.250397690690072950000431855143.mhd',).

Traceback (most recent call last): File "/net/ascratch/people/plgztabor/MONAI/luna16_prepare_images.py", line 122, in main() File "/net/ascratch/people/plgztabor/MONAI/luna16_prepare_images.py", line 110, in main for batch_data in process_loader: 2022-12-12 14:07:16,167 - NrrdReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.287966244644280690737019247886.mhd',).

File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 628, in next data = self._next_data() File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data 2022-12-12 14:07:16,167 - PydicomReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.287966244644280690737019247886.mhd',).

return self._process_data(data)

File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data data.reraise() File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/torch/_utils.py", line 543, in reraise 2022-12-12 14:07:16,169 - NibabelReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.768276876111112560631432843476.mhd',).

2022-12-12 14:07:16,169 - PILReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.768276876111112560631432843476.mhd',).

2022-12-12 14:07:16,169 - NumpyReader: unable to load ('/net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.768276876111112560631432843476.mhd',).

raise exception

RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/monai/transforms/transform.py", line 91, in apply_transform return _apply_transform(transform, data, unpack_items) File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/monai/transforms/transform.py", line 55, in _apply_transform return transform(parameters) File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/monai/transforms/io/dictionary.py", line 154, in call data = self._loader(d[key], reader) File "/net/software/testing/software/monai/1.0.1/lib/python3.10/site-packages/monai/transforms/io/array.py", line 258, in call raise RuntimeError( RuntimeError: LoadImage cannot find a suitable reader for file: /net/ascratch/people/plgztabor/MONAI/DATA/1.3.6.1.4.1.14519.5.2.1.6279.6001.250397690690072950000431855143.mhd. Please install the reader libraries, see also the installation instructions: https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies. The current registered: [<monai.data.image_reader.PydicomReader object at 0x14c220d351e0>, <monai.data.image_reader.NrrdReader object at 0x14c220d34820>, <monai.data.image_reader.NumpyReader object at 0x14c220d34e20>, <monai.data.image_reader.PILReader object at 0x14c220d35990>, <monai.data.image_reader.NibabelReader object at 0x14c220d35570>].

Can-Zhao commented 1 year ago

Thank you for reaching out!

  1. LUNA16 dataset excluded nodules <3mm, that's why the actual used files are much less than 888. https://luna16.grand-challenge.org/Data/#:~:text=The%20reference%20standard%20of%20our%20challenge%20consists%20of%20all%20nodules%20%3E%3D%203%20mm%20accepted%20by%20at%20least%203%20out%20of%204%20radiologists.

  2. The I/O issue seems to comes from ITKreader. Could you try pip install itk>=5.2 and try it again?

Thanks!

taborzbislaw commented 1 year ago

Thank you very much. After installing itk>=5.2 luna16_prepare_images.py works as expected.