search

Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.93k stars 1.09k forks source link

test error data_analyzer_byoc.ipynb #5951

Closed wyli closed 1 year ago

wyli commented 1 year ago

Describe the bug

[2023-02-07T01:05:27.319Z] Running ./auto3dseg/notebooks/data_analyzer_byoc.ipynb
[2023-02-07T01:05:27.319Z] Checking PEP8 compliance...
[2023-02-07T01:05:28.247Z] Running notebook...
[2023-02-07T01:05:32.408Z] MONAI version: 1.1.0+54.g385ac0bb
[2023-02-07T01:05:32.408Z] Numpy version: 1.22.2
[2023-02-07T01:05:32.408Z] Pytorch version: 1.10.2+cu102
[2023-02-07T01:05:32.408Z] MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
[2023-02-07T01:05:32.408Z] MONAI rev id: 385ac0bb6612f88f92e0862a6aee033fb8df5de7
[2023-02-07T01:05:32.408Z] MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
[2023-02-07T01:05:32.408Z] 
[2023-02-07T01:05:32.408Z] Optional dependencies:
[2023-02-07T01:05:32.408Z] Pytorch Ignite version: 0.4.10
[2023-02-07T01:05:32.408Z] ITK version: 5.3.0
[2023-02-07T01:05:32.408Z] Nibabel version: 5.0.0
[2023-02-07T01:05:32.408Z] scikit-image version: 0.19.3
[2023-02-07T01:05:32.408Z] Pillow version: 7.0.0
[2023-02-07T01:05:32.408Z] Tensorboard version: 2.9.0
[2023-02-07T01:05:32.408Z] gdown version: 4.6.0
[2023-02-07T01:05:32.408Z] TorchVision version: 0.11.3+cu102
[2023-02-07T01:05:32.408Z] tqdm version: 4.64.1
[2023-02-07T01:05:32.408Z] lmdb version: 1.4.0
[2023-02-07T01:05:32.408Z] psutil version: 5.9.4
[2023-02-07T01:05:32.408Z] pandas version: 1.1.5
[2023-02-07T01:05:32.408Z] einops version: 0.6.0
[2023-02-07T01:05:32.408Z] transformers version: 4.21.3
[2023-02-07T01:05:32.408Z] mlflow version: 2.1.1
[2023-02-07T01:05:32.408Z] pynrrd version: 1.0.0
[2023-02-07T01:05:32.408Z] 
[2023-02-07T01:05:32.409Z] For details about installing the optional dependencies, please visit:
[2023-02-07T01:05:32.409Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2023-02-07T01:05:32.409Z] 
[2023-02-07T01:05:34.298Z] papermill  --progress-bar -k python3
[2023-02-07T01:05:34.298Z] /usr/local/lib/python3.8/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-02-07T01:05:34.298Z]   warnings.warn(
[2023-02-07T01:06:00.453Z] 
Executing:   0%|          | 0/15 [00:00<?, ?cell/s]
Executing:   7%|▋         | 1/15 [00:01<00:17,  1.26s/cell]
Executing:  20%|██        | 3/15 [00:06<00:26,  2.19s/cell]
Executing:  33%|███▎      | 5/15 [00:10<00:20,  2.09s/cell]
Executing:  47%|████▋     | 7/15 [00:10<00:10,  1.26s/cell]
Executing:  67%|██████▋   | 10/15 [00:24<00:14,  2.94s/cell]
Executing:  67%|██████▋   | 10/15 [00:26<00:13,  2.61s/cell]
[2023-02-07T01:06:00.453Z] /usr/local/lib/python3.8/dist-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
[2023-02-07T01:06:00.453Z]   warnings.warn(
[2023-02-07T01:06:00.453Z] Traceback (most recent call last):
[2023-02-07T01:06:00.453Z]   File "/usr/local/bin/papermill", line 8, in <module>
[2023-02-07T01:06:00.453Z]     sys.exit(papermill())
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1130, in __call__
[2023-02-07T01:06:00.453Z]     return self.main(*args, **kwargs)
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1055, in main
[2023-02-07T01:06:00.453Z]     rv = self.invoke(ctx)
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke
[2023-02-07T01:06:00.453Z]     return ctx.invoke(self.callback, **ctx.params)
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 760, in invoke
[2023-02-07T01:06:00.453Z]     return __callback(*args, **kwargs)
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 26, in new_func
[2023-02-07T01:06:00.453Z]     return f(get_current_context(), *args, **kwargs)
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/papermill/cli.py", line 250, in papermill
[2023-02-07T01:06:00.453Z]     execute_notebook(
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/papermill/execute.py", line 128, in execute_notebook
[2023-02-07T01:06:00.453Z]     raise_for_execution_errors(nb, output_path)
[2023-02-07T01:06:00.453Z]   File "/usr/local/lib/python3.8/dist-packages/papermill/execute.py", line 232, in raise_for_execution_errors
[2023-02-07T01:06:00.453Z]     raise error
[2023-02-07T01:06:00.453Z] papermill.exceptions.PapermillExecutionError: 
[2023-02-07T01:06:00.453Z] ---------------------------------------------------------------------------
[2023-02-07T01:06:00.453Z] Exception encountered at "In [5]":
[2023-02-07T01:06:00.453Z] ---------------------------------------------------------------------------
[2023-02-07T01:06:00.453Z] KeyError                                  Traceback (most recent call last)
[2023-02-07T01:06:00.453Z] Cell In[5], line 37
[2023-02-07T01:06:00.453Z]      33     result[DataStatsKeys.SUMMARY] = summarizer.summarize(result[DataStatsKeys.BY_CASE])
[2023-02-07T01:06:00.453Z]      34     return result
[2023-02-07T01:06:00.453Z] ---> 37 result = my_analyzer(sim_datalist, sim_dataroot, summarizer)
[2023-02-07T01:06:00.453Z] 
[2023-02-07T01:06:00.453Z] Cell In[5], line 33, in my_analyzer(datalist, dataroot, my_summarizer)
[2023-02-07T01:06:00.453Z]      23     stats_by_cases = {
[2023-02-07T01:06:00.453Z]      24         DataStatsKeys.BY_CASE_IMAGE_PATH: d[DataStatsKeys.BY_CASE_IMAGE_PATH],
[2023-02-07T01:06:00.453Z]      25         DataStatsKeys.BY_CASE_LABEL_PATH: d[DataStatsKeys.BY_CASE_LABEL_PATH],
[2023-02-07T01:06:00.453Z]    (...)
[2023-02-07T01:06:00.453Z]      29         "user_stats": d["user_stats"],
[2023-02-07T01:06:00.453Z]      30     }
[2023-02-07T01:06:00.453Z]      32 result[DataStatsKeys.BY_CASE].append(stats_by_cases)
[2023-02-07T01:06:00.454Z] ---> 33 result[DataStatsKeys.SUMMARY] = summarizer.summarize(result[DataStatsKeys.BY_CASE])
[2023-02-07T01:06:00.454Z]      34 return result
[2023-02-07T01:06:00.454Z] 
[2023-02-07T01:06:00.454Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/auto3dseg/seg_summarizer.py:211, in SegSummarizer.summarize(self, data)
[2023-02-07T01:06:00.454Z]     209 for analyzer in self.summary_analyzers:
[2023-02-07T01:06:00.454Z]     210     if callable(analyzer):
[2023-02-07T01:06:00.454Z] --> 211         report.update({analyzer.stats_name: analyzer(data)})
[2023-02-07T01:06:00.454Z]     213 return report
[2023-02-07T01:06:00.454Z] 
[2023-02-07T01:06:00.454Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/auto3dseg/analyzer.py:991, in ImageHistogramSumm.__call__(self, data)
[2023-02-07T01:06:00.454Z]     988     raise ValueError(f"Callable {self.__class__} input list is empty")
[2023-02-07T01:06:00.454Z]     990 if self.stats_name not in data[0]:
[2023-02-07T01:06:00.454Z] --> 991     raise KeyError(f"{self.stats_name} is not in input data")
[2023-02-07T01:06:00.454Z]     993 summ_histogram: dict = {}
[2023-02-07T01:06:00.454Z]     995 for d in data:
[2023-02-07T01:06:00.454Z] 
[2023-02-07T01:06:00.454Z] KeyError: 'image_histogram is not in input data'

introduced by this commit https://github.com/Project-MONAI/MONAI/commit/f9842c0dda70714c826eeb8b12bbf93d987a0584

hi @mingxin-zheng, do you have time to have a look?

cc @Nic-Ma @Shadow-Devil

Shadow-Devil commented 1 year ago

This was the place where I changed it from returning the Error to raising the Error. Before this commit, the validation was just ignored. https://github.com/Project-MONAI/MONAI/blob/385ac0bb6612f88f92e0862a6aee033fb8df5de7/monai/auto3dseg/analyzer.py#L990-L991

mingxin-zheng commented 1 year ago

Hi @wyli @Shadow-Devil thank you for raising it. I would like to add @holgerroth to the discussion because the analyzer is for FL.

mingxin-zheng commented 1 year ago

On a second look, this notebook should not be using any components in ImageHistogram. Calling ImageHistogram seems a bug to me. Let me take a look and report back later.