search

openvinotoolkit / anomalib

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.
https://anomalib.readthedocs.io/en/latest/
Apache License 2.0
3.4k stars 615 forks source link

[Bug]: RuntimeError("Please call `iter(combined_loader)` first.") #2078

Open albert-mg opened 1 month ago

albert-mg commented 1 month ago

Describe the bug

GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\core\optimizer.py:181: `LightningModule.configure_optimizers` returned `None`, this fit will run with no optimizer

  | Name                  | Type                     | Params
-------------------------------------------------------------------
0 | model                 | PatchcoreModel           | 24.9 M
1 | _transform            | Compose                  | 0     
2 | normalization_metrics | MinMax                   | 0     
3 | image_threshold       | F1AdaptiveThreshold      | 0     
4 | pixel_threshold       | F1AdaptiveThreshold      | 0     
5 | image_metrics         | AnomalibMetricCollection | 0     
6 | pixel_metrics         | AnomalibMetricCollection | 0     
-------------------------------------------------------------------
24.9 M    Trainable params
0         Non-trainable params
24.9 M    Total params
99.450    Total estimated model params size (MB)
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\connectors\data_connector.py:436: Consider setting `persistent_workers=True` in 'train_dataloader' to speed up the dataloader worker initialization.
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\core\optimizer.py:181: `LightningModule.configure_optimizers` returned `None`, this fit will run with no optimizer

  | Name                  | Type                     | Params
-------------------------------------------------------------------
0 | model                 | PatchcoreModel           | 24.9 M
1 | _transform            | Compose                  | 0
2 | normalization_metrics | MinMax                   | 0
3 | image_threshold       | F1AdaptiveThreshold      | 0
4 | pixel_threshold       | F1AdaptiveThreshold      | 0
5 | image_metrics         | AnomalibMetricCollection | 0
6 | pixel_metrics         | AnomalibMetricCollection | 0
-------------------------------------------------------------------
24.9 M    Trainable params
0         Non-trainable params
24.9 M    Total params
99.450    Total estimated model params size (MB)
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\connectors\data_connector.py:436: Consider setting `persistent_workers=True` in 'train_dataloader' to speed up the dataloader worker initialization.
Traceback (most recent call last):
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\call.py", line 44, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 580, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 987, in _run
    results = self._run_stage()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 1033, in _run_stage
    self.fit_loop.run()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 197, in run
    self.setup_data()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 263, in setup_data
    iter(self._data_fetcher)  # creates the iterator inside the fetcher
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 104, in __iter__
    super().__iter__()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 51, in __iter__
    self.iterator = iter(self.combined_loader)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 351, in __iter__
    iter(iterator)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 92, in __iter__
    super().__iter__()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 43, in __iter__
    self.iterators = [iter(iterable) for iterable in self.iterables]
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 43, in <listcomp>
    self.iterators = [iter(iterable) for iterable in self.iterables]
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\torch\utils\data\dataloader.py", line 439, in __iter__
    return self._get_iterator()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\torch\utils\data\dataloader.py", line 387, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\torch\utils\data\dataloader.py", line 1040, in __init__
    w.start()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\context.py", line 336, in _Popen
    return Popen(process_obj)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\runpy.py", line 289, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Users\3855445\sorting\anomalib\trainer.py", line 12, in <module>
    engine.fit(datamodule=datamodule, model=model)
  File "C:\Users\3855445\sorting\anomalib\src\anomalib\engine\engine.py", line 541, in fit
    self.trainer.fit(model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 544, in fit
    call._call_and_handle_interrupt(
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\call.py", line 68, in _call_and_handle_interrupt
    trainer._teardown()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 1014, in _teardown
    loop.teardown()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 411, in teardown
    self._data_fetcher.teardown()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 79, in teardown
    self.reset()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 141, in reset
    super().reset()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 75, in reset
    self.length = sized_len(self.combined_loader)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\fabric\utilities\data.py", line 51, in sized_len
    length = len(dataloader)  # type: ignore [arg-type]
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 358, in __len__
    raise RuntimeError("Please call `iter(combined_loader)` first.")
RuntimeError: Please call `iter(combined_loader)` first.

Dataset

MVTec

Model

PatchCore

Steps to reproduce the behavior

# Import the required modules
from anomalib.data import MVTec
from anomalib.models import Patchcore
from anomalib.engine import Engine

# Initialize the datamodule, model and engine
datamodule = MVTec()
model = Patchcore()
engine = Engine()

# Train the model
engine.fit(datamodule=datamodule, model=model)

OS information

OS information:

Expected behavior

follow the Installation and Training,but some runtime bug appear

Screenshots

No response

Pip/GitHub

pip

What version/branch did you use?

main

Configuration YAML

default

Logs

GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\core\optimizer.py:181: `LightningModule.configure_optimizers` returned `None`, this fit will run with no optimizer

  | Name                  | Type                     | Params
-------------------------------------------------------------------
0 | model                 | PatchcoreModel           | 24.9 M
1 | _transform            | Compose                  | 0     
2 | normalization_metrics | MinMax                   | 0     
3 | image_threshold       | F1AdaptiveThreshold      | 0     
4 | pixel_threshold       | F1AdaptiveThreshold      | 0     
5 | image_metrics         | AnomalibMetricCollection | 0     
6 | pixel_metrics         | AnomalibMetricCollection | 0     
-------------------------------------------------------------------
24.9 M    Trainable params
0         Non-trainable params
24.9 M    Total params
99.450    Total estimated model params size (MB)
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\connectors\data_connector.py:436: Consider setting `persistent_workers=True` in 'train_dataloader' to speed up the dataloader worker initialization.
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\core\optimizer.py:181: `LightningModule.configure_optimizers` returned `None`, this fit will run with no optimizer

  | Name                  | Type                     | Params
-------------------------------------------------------------------
0 | model                 | PatchcoreModel           | 24.9 M
1 | _transform            | Compose                  | 0
2 | normalization_metrics | MinMax                   | 0
3 | image_threshold       | F1AdaptiveThreshold      | 0
4 | pixel_threshold       | F1AdaptiveThreshold      | 0
5 | image_metrics         | AnomalibMetricCollection | 0
6 | pixel_metrics         | AnomalibMetricCollection | 0
-------------------------------------------------------------------
24.9 M    Trainable params
0         Non-trainable params
24.9 M    Total params
99.450    Total estimated model params size (MB)
C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\connectors\data_connector.py:436: Consider setting `persistent_workers=True` in 'train_dataloader' to speed up the dataloader worker initialization.
Traceback (most recent call last):
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\call.py", line 44, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 580, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 987, in _run
    results = self._run_stage()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 1033, in _run_stage
    self.fit_loop.run()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 197, in run
    self.setup_data()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 263, in setup_data
    iter(self._data_fetcher)  # creates the iterator inside the fetcher
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 104, in __iter__
    super().__iter__()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 51, in __iter__
    self.iterator = iter(self.combined_loader)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 351, in __iter__
    iter(iterator)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 92, in __iter__
    super().__iter__()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 43, in __iter__
    self.iterators = [iter(iterable) for iterable in self.iterables]
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 43, in <listcomp>
    self.iterators = [iter(iterable) for iterable in self.iterables]
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\torch\utils\data\dataloader.py", line 439, in __iter__
    return self._get_iterator()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\torch\utils\data\dataloader.py", line 387, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\torch\utils\data\dataloader.py", line 1040, in __init__
    w.start()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\context.py", line 336, in _Popen
    return Popen(process_obj)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\runpy.py", line 289, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Users\3855445\sorting\anomalib\trainer.py", line 12, in <module>
    engine.fit(datamodule=datamodule, model=model)
  File "C:\Users\3855445\sorting\anomalib\src\anomalib\engine\engine.py", line 541, in fit
    self.trainer.fit(model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 544, in fit
    call._call_and_handle_interrupt(
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\call.py", line 68, in _call_and_handle_interrupt
    trainer._teardown()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 1014, in _teardown
    loop.teardown()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 411, in teardown
    self._data_fetcher.teardown()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 79, in teardown
    self.reset()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 141, in reset
    super().reset()
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 75, in reset
    self.length = sized_len(self.combined_loader)
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\fabric\utilities\data.py", line 51, in sized_len
    length = len(dataloader)  # type: ignore [arg-type]
  File "C:\Users\3855445\.conda\envs\anomalib_env\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 358, in __len__
    raise RuntimeError("Please call `iter(combined_loader)` first.")
RuntimeError: Please call `iter(combined_loader)` first.

Code of Conduct

samet-akcay commented 1 month ago

do you want to run this on CPU?

alexriedel1 commented 1 month ago

How are you running the code? In a Jupyter Notebook, in a Python script? Whats the actual code you are running? On which machine? Did you try to do what the error exception told you to do, that is running your code inside a if __name__ == '__main__': loop ? Did you try to set the number of dataloader workers to 0 in your config file?