search

openvinotoolkit / anomalib

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.
https://anomalib.readthedocs.io/en/latest/
Apache License 2.0
3.41k stars 616 forks source link

How to avoid Runtime Error #2071

Closed microchila closed 1 month ago

microchila commented 1 month ago

What is the motivation for this task?

A runtime error occurs when running the following code：

from anomalib.data import MVTec
from anomalib.models import Patchcore
from anomalib.engine import Engine

datamodule = MVTec()
model = Patchcore()
engine = Engine()

engine.fit(datamodule=datamodule, model=model)

Describe the solution you'd like

Normal training

Additional context

Contents of the error message：

GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
You are using a CUDA device ('NVIDIA GeForce RTX 3060 Laptop GPU') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\core\optimizer.py:181: `LightningModule.configure_optimizers` returned `None`, this fit will run with no optimizer

  | Name                  | Type                     | Params
-------------------------------------------------------------------
0 | model                 | PatchcoreModel           | 24.9 M
1 | _transform            | Compose                  | 0     
2 | normalization_metrics | MinMax                   | 0     
3 | image_threshold       | F1AdaptiveThreshold      | 0     
4 | pixel_threshold       | F1AdaptiveThreshold      | 0     
5 | image_metrics         | AnomalibMetricCollection | 0     
6 | pixel_metrics         | AnomalibMetricCollection | 0     
-------------------------------------------------------------------
24.9 M    Trainable params
0         Non-trainable params
24.9 M    Total params
99.450    Total estimated model params size (MB)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
You are using a CUDA device ('NVIDIA GeForce RTX 3060 Laptop GPU') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
F1Score class exists for backwards compatibility. It will be removed in v1.1. Please use BinaryF1Score from torchmetrics instead
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\core\optimizer.py:181: `LightningModule.configure_optimizers` returned `None`, this fit will run with no optimizer

  | Name                  | Type                     | Params
-------------------------------------------------------------------
0 | model                 | PatchcoreModel           | 24.9 M
1 | _transform            | Compose                  | 0     
2 | normalization_metrics | MinMax                   | 0     
3 | image_threshold       | F1AdaptiveThreshold      | 0     
4 | pixel_threshold       | F1AdaptiveThreshold      | 0     
5 | image_metrics         | AnomalibMetricCollection | 0     
6 | pixel_metrics         | AnomalibMetricCollection | 0     
-------------------------------------------------------------------
24.9 M    Trainable params
0         Non-trainable params
24.9 M    Total params
99.450    Total estimated model params size (MB)
Traceback (most recent call last):
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\trainer\call.py", line 44, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 580, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 987, in _run
    results = self._run_stage()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 1033, in _run_stage
    self.fit_loop.run()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 197, in run
    self.setup_data()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 263, in setup_data
    iter(self._data_fetcher)  # creates the iterator inside the fetcher
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 104, in __iter__
    super().__iter__()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 51, in __iter__
    self.iterator = iter(self.combined_loader)
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 351, in __iter__
    iter(iterator)
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 92, in __iter__
    super().__iter__()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 43, in __iter__
    self.iterators = [iter(iterable) for iterable in self.iterables]
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 43, in <listcomp>
    self.iterators = [iter(iterable) for iterable in self.iterables]
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\torch\utils\data\dataloader.py", line 434, in __iter__
    self._iterator = self._get_iterator()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\torch\utils\data\dataloader.py", line 387, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\torch\utils\data\dataloader.py", line 1040, in __init__
    w.start()
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\context.py", line 336, in _Popen
    return Popen(process_obj)
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "D:\software\anaconda\envs\anomalib1\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "D:\software\anaconda\envs\anomalib1\lib\runpy.py", line 289, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "D:\software\anaconda\envs\anomalib1\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "D:\software\anaconda\envs\anomalib1\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Leo\Desktop\anomalib-main\anomalib\1.py", line 12, in <module>
    engine.fit(datamodule=datamodule, model=model)
  File "C:\Users\Leo\Desktop\anomalib-main\anomalib\src\anomalib\engine\engine.py", line 540, in fit
    self.trainer.fit(model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 544, in fit
    call._call_and_handle_interrupt(
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\trainer\call.py", line 68, in _call_and_handle_interrupt
    trainer._teardown()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\trainer\trainer.py", line 1014, in _teardown
    loop.teardown()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fit_loop.py", line 411, in teardown
    self._data_fetcher.teardown()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 79, in teardown
    self.reset()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 141, in reset
    super().reset()
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\loops\fetchers.py", line 75, in reset
    self.length = sized_len(self.combined_loader)
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\fabric\utilities\data.py", line 51, in sized_len
    length = len(dataloader)  # type: ignore [arg-type]
  File "D:\software\anaconda\envs\anomalib1\lib\site-packages\lightning\pytorch\utilities\combined_loader.py", line 358, in __len__
    raise RuntimeError("Please call `iter(combined_loader)` first.")
RuntimeError: Please call `iter(combined_loader)` first.
blaz-r commented 1 month ago

I believe to solve this you need to write the code inside main like so:

...imports..

def main():
  datamodule = MVTec()
  model = Patchcore()
  engine = Engine()

  engine.fit(datamodule=datamodule, model=model)

if __name__ == "__main__":
   main()