Project-MONAI / MONAILabel

MONAI Label is an intelligent open source image labeling and learning tool.
https://docs.monai.io/projects/label
Apache License 2.0
610 stars 196 forks source link

Not able to train model using monailabel0.4.0rc and the spleen dataset #718

Closed karenhes closed 2 years ago

karenhes commented 2 years ago

I followed the following description to install and start running monailabel:

pip install monailabel==0.4.0rc monailabel apps --download --name radiology --output apps monailabel datasets --download --name Task09_Spleen --output datasets monailabel start_server --app apps/radiology --studies datasets/Task09_Spleen/imagesTr --conf models deepedit

Then I opened Slicer and imported and submitted some labels. Then I chose deepedit and clicked Train. Unfortunately, the process crashed and gave the following output in the terminal:

[2022-04-01 12:53:17,374] [20720] [MainThread] [INFO] (ignite.engine.engine.SupervisedTrainer:138) - Restored all variables from C:\Users\Karen\Documents\monailabel-folder\spleenV2\radiology\model\pretrained_deepedit_dynunet.pt [2022-04-01 12:53:20,154] [20720] [MainThread] [ERROR] (ignite.engine.engine.SupervisedTrainer:853) - Current run is terminating due to exception: applying transform <monai.transforms.compose.Compose object at 0x00000289860C3670> [2022-04-01 12:53:20,154] [20720] [MainThread] [ERROR] (ignite.engine.engine.SupervisedTrainer:172) - Exception: applying transform <monai.transforms.compose.Compose object at 0x00000289860C3670> Traceback (most recent call last): File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\transforms\transform.py", line 89, in apply_transform return _apply_transform(transform, data, unpack_items) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\transforms\transform.py", line 53, in _apply_transform return transform(parameters) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monailabel\deepedit\multilabel\transforms.py", line 757, in call for idx, (keylabel, ) in enumerate(d["label_names"].items()): KeyError: 'label_names' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\transforms\transform.py", line 89, in apply_transform return _apply_transform(transform, data, unpack_items) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\transforms\transform.py", line 53, in _applytransform return transform(parameters) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\transforms\compose.py", line 173, in call input = apply_transform(transform, input, self.map_items, self.unpack_items, self.log_stats) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\transforms\transform.py", line 113, in apply_transform raise RuntimeError(f"applying transform {transform}") from e RuntimeError: applying transform <monailabel.deepedit.multilabel.transforms.SplitPredsLabeld object at 0x00000289860C3850> The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\ignite\engine\engine.py", line 840, in _run_once_on_dataset self.state.output = self._process_function(self, self.state.batch) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monailabel\deepedit\multilabel\interaction.py", line 98, in call return engine._iteration(engine, batchdata) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\engines\trainer.py", line 214, in _iteration engine.fire_event(IterationEvents.MODEL_COMPLETED) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\ignite\engine\engine.py", line 445, in fire_event return self._fire_event(event_name) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\ignite\engine\engine.py", line 421, in _fire_event func(first, (event_args + others), **kwargs) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\engines\workflow.py", line 226, in _run_postprocessing engine.state.batch[i], engine.state.output[i] = engine_apply_transform(b, o, posttrans) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\engines\utils.py", line 219, in engine_apply_transform transformed_data = apply_transform(transform, data) File "C:\Users\Karen\anaconda3\envs\monai-env\lib\site-packages\monai\transforms\transform.py", line 113, in apply_transform raise RuntimeError(f"applying transform {transform}") from e RuntimeError: applying transform <monai.transforms.compose.Compose object at 0x00000289860C3670> [2022-04-01 12:53:20,214] [20720] [MainThread] [ERROR] (ignite.engine.engine.SupervisedTrainer:782) - Engine run is terminating due to exception: applying transform <monai.transforms.compose.Compose object at 0x00000289860C3670>

Would have been great to get some help with this issue!!

SachidanandAlle commented 2 years ago

Try simple spleen segmentation model.. may be you don't have enough gpu

diazandr3s commented 2 years ago

Thanks for raising this error, @karenhes. We've recently fixed this issue. Can you please install the latest monailabel-weekly and let us know how that goes?

SachidanandAlle commented 2 years ago

any updates on this? I guess you forgot to add the full exception in the log.. it just ends at: Engine run is terminating due to exception: applying transform <monai.transforms.compose.Compose object at 0x00000289860C3670>

full exception trace?

karenhes commented 2 years ago

It may be that I missed parts of the log. However, after installing an updated version everything is working out of the box for the spleen dataset and the smartedit function is improved :) (unfortunately I cannot figure out how to train my own data on other organs as I did with the stable version, but that is next step.) I installed a new version as follows:

pip install git+https://github.com/Project-MONAI/MONAILabel#egg=monailabel monailabel apps --download --name radiology --output apps monailabel datasets --download --name Task09_Spleen --output datasets monailabel start_server --app apps/radiology --studies datasets/Task09_Spleen/imagesTr --conf models deepedit

diazandr3s commented 2 years ago

Thanks for letting us know, @karenhes