Project-MONAI / model-zoo

MONAI Model Zoo that hosts models in the MONAI Bundle format.
Apache License 2.0
182 stars 68 forks source link

Error using inference in `Pancreas ct dints segmentation` model #403

Closed IvanHCenalmor closed 1 year ago

IvanHCenalmor commented 1 year ago

Describe the bug I was testing the inference in the Pancreas ct dints segmentation model https://github.com/Project-MONAI/model-zoo/tree/dev/models/pancreas_ct_dints_segmentation and I received the following error:

Traceback (most recent call last):
  File "C:\Users\Administrador\anaconda3\envs\monai\lib\site-packages\ignite\engine\engine.py", line 946, in _internal_run_as_gen
    self._fire_event(Events.STARTED)
  File "C:\Users\Administrador\anaconda3\envs\monai\lib\site-packages\ignite\engine\engine.py", line 425, in _fire_event
    func(*first, *(event_args + others), **kwargs)
  File "C:\Users\Administrador\anaconda3\envs\monai\lib\site-packages\monai\handlers\checkpoint_loader.py", line 128, in __call__
    Checkpoint.load_objects(to_load=self.load_dict, checkpoint=checkpoint, strict=self.strict)
  File "C:\Users\Administrador\anaconda3\envs\monai\lib\site-packages\ignite\handlers\checkpoint.py", line 635, in load_objects
    _load_object(obj, checkpoint_obj[k])
  File "C:\Users\Administrador\anaconda3\envs\monai\lib\site-packages\ignite\handlers\checkpoint.py", line 620, in _load_object
    obj.load_state_dict(chkpt_obj, strict=is_state_dict_strict)
  File "C:\Users\Administrador\anaconda3\envs\monai\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DiNTS:
        Missing key(s) in state_dict: "dints_space.cell_tree.(0, 1).op.ops.2.conv.weight",...

I think the problem is in the provided /models/search_code_18590.pt, their keys do not match with the actual definition of the DiNTS model.

To Reproduce Steps to reproduce the behavior:

  1. Go to https://github.com/Project-MONAI/model-zoo/tree/dev/models/pancreas_ct_dints_segmentation
  2. Download the model and install the MONAI Python library
  3. Run commands python -m monai.bundle run --config_file configs/inference.yaml

Expected behavior In my case I was only testing, I am more interested in the model than in using the inference, and at least I was expecting to be able to load the model.

Environment

Ensuring you use the relevant python executable, please paste the output of:

================================
Printing MONAI config...
================================
MONAI version: 1.1.0
Numpy version: 1.24.3
Pytorch version: 2.0.0
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: a2ec3752f54bfc3b40e7952234fbeb5452ed63e3
MONAI __file__: C:\Users\Administrador\anaconda3\envs\monai\lib\site-packages\monai\__init__.py

Optional dependencies:
Pytorch Ignite version: 0.4.10
Nibabel version: 5.1.0
scikit-image version: 0.20.0
Pillow version: 9.4.0
Tensorboard version: 2.12.2
gdown version: 4.7.1
TorchVision version: 0.15.0
tqdm version: 4.65.0
lmdb version: 1.4.1
psutil version: 5.9.0
pandas version: 2.0.1
einops version: 0.6.1
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: 2.3.1
pynrrd version: 0.4.3

For details about installing the optional dependencies, please visit:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================
Printing system config...
================================
System: Windows
Win32 version: ('10', '10.0.22000', 'SP0', 'Multiprocessor Free')
Win32 edition: Professional
Platform: Windows-10-10.0.22000-SP0
Processor: Intel64 Family 6 Model 183 Stepping 1, GenuineIntel
Machine: AMD64
Python version: 3.9.16
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Num physical CPUs: 24
Num logical CPUs: 32
Num usable CPUs: 32
CPU usage (%): [0.0, 0.2, 0.2, 0.1, 17.7, 1.3, 0.4, 6.7, 0.1, 0.1, 0.1, 0.3, 0.0, 0.0, 0.0, 0.0, 0.5, 9.2, 1.1, 0.1, 3.6, 4.8, 0.7, 0.2, 7.0, 0.1, 0.4, 0.0, 6.2, 0.0, 0.0, 0.1]
CPU freq. (MHz): 3000
Load avg. in last 1, 5, 15 mins (%): [0.0, 0.0, 0.0]
Disk usage (%): 0.5
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 127.8
Available memory (GB): 115.9
Used memory (GB): 11.8

================================
Printing GPU config...
================================
Num GPUs: 1
Has CUDA: True
CUDA version: 11.8
cuDNN enabled: True
cuDNN version: 8700
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_61', 'sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90', 'compute_37']
GPU 0 Name: NVIDIA GeForce RTX 4090
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 128
GPU 0 Total memory (GB): 24.0
GPU 0 CUDA capability (maj.min): 8.9
Nic-Ma commented 1 year ago

Hi @dongyang0122 ,

Could you please help take a look at this question?

Thanks in advance.

yiheng-wang-nv commented 1 year ago

MONAI version: 1.1.0

Hi @IvanHCenalmor , the network that this bundle used is changed after MONAI 1.1, could you please try to upgrade your MONAI version and recheck it? You can try the following command:

pip install monai==1.2.0rc5
dongyang0122 commented 1 year ago

Hi @IvanHCenalmor, the most recent DiNTS bundle is founded on the version of MONAI subsequent to 1.1.0. You have the option to utilize either the aforementioned version or the latest development branch in GitHub, accessible through installation.

yiheng-wang-nv commented 1 year ago

close this issue as the error should be fixed with new monai version aforementioned. Feel free to reopen it if still having the issue.