search

openvinotoolkit / anomalib

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.
https://anomalib.readthedocs.io/en/latest/
Apache License 2.0
3.68k stars 654 forks source link

[Bug]: Unable to use costumized dataset following anomalib tutorial #1215

Closed cugwu closed 1 year ago

cugwu commented 1 year ago

Describe the bug

I want to use a customized dataset and train reverse distillation. My dataset doesn't have binary masks. I followed the anomalib tutorial in: https://openvinotoolkit.github.io/anomalib/how_to_guides/train_custom_data.html But I keep getting the error: File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/data/folder.py", line 59, in make_folder_dataset assert normal_dir.is_dir(), "A folder location must be provided in normal_dir." AssertionError: A folder location must be provided in normal_dir.

Even if I'm providing a folder as input to the parameters. I also tried with the hazelnut_toy dataset provided by anomalib and I keep getting the same error

Dataset

Other (please specify in the text field below)

Model

Reverse Distillation

Steps to reproduce the behavior

I followed the tutorial in: https://openvinotoolkit.github.io/anomalib/how_to_guides/train_custom_data.html

  1. I took the config.yaml of the model reverse distillation
  2. I modified the dataset config in order to use a customized dataset
  3. I keep getting the following error: AssertionError: A folder location must be provided in normal_dir even if I'm giving folders as input

I also try to use hazelnut_toy dataset to recreate the same configuration of the tutorial and I keep getting the same error.

OS information

OS information:

Expected behavior

I was expecting to train revers distillation with my customized dataset instead it seems like the code is not able to recognize that I'm giving a folder as input

Screenshots

No response

Pip/GitHub

pip

What version/branch did you use?

I used the following command: pip install anomalib[full]

Configuration YAML

dataset:
    name: hazelnut
    format: folder
    root: ./datasets/hazelnut_toy
    normal_dir: good # name of the folder containing normal images.
    abnormal_dir: colour # name of the folder containing abnormal images.
    task: classification # classification or segmentation
    mask_dir: null #optional
    normal_test_dir: null # optional
    extensions: null
    split_ratio: 0.2  # normal images ratio to create a test split
    seed: 0
    image_size: 256
    train_batch_size: 32
    eval_batch_size: 32
    num_workers: 8
    normalization: imagenet # data distribution to which the images will be normalized
    transform_config:
      train: null
      eval: null
    test_split_mode: from_dir # options: [from_dir, synthetic]
    test_split_ratio: 0.2 # fraction of train images held out testing (usage depends on test_split_mode)
    val_split_mode: same_as_test # options: [same_as_test, from_test, synthetic]
    val_split_ratio: 0.5 # fraction of train/test images held out for validation (usage depends on val_split_mode)
    tiling:
      apply: false
      tile_size: 64
      stride: null
      remove_border_count: 0
      use_random_tiling: False
      random_tile_count: 16

model:
  name: reverse_distillation
  lr: 0.005
  backbone: wide_resnet50_2
  pre_trained: true
  layers:
    - layer1
    - layer2
    - layer3
  early_stopping:
    patience: 3
    metric: pixel_AUROC
    mode: max
  beta1: 0.5
  beta2: 0.99
  normalization_method: min_max # options: [null, min_max, cdf]
  anomaly_map_mode: multiply

metrics:
  image:
    - F1Score
    - AUROC
  pixel:
    - F1Score
    - AUROC
  threshold:
    method: adaptive #options: [adaptive, manual]
    manual_image: null
    manual_pixel: null

visualization:
  show_images: False # show images on the screen
  save_images: True # save images to the file system
  log_images: True # log images to the available loggers (if any)
  image_save_path: null # path to which images will be saved
  mode: full # options: ["full", "simple"]

project:
  seed: 42
  path: ./results

logging:
  logger: [] # options: [comet, tensorboard, wandb, csv] or combinations.
  log_graph: false # Logs the model graph to respective logger.

optimization:
  export_mode: null # options: torch, onnx, openvino

# PL Trainer Args. Don't add extra parameter here.
trainer:
  enable_checkpointing: true
  default_root_dir: null
  gradient_clip_val: 0
  gradient_clip_algorithm: norm
  num_nodes: 1
  devices: 1
  enable_progress_bar: true
  overfit_batches: 0.0
  track_grad_norm: -1
  check_val_every_n_epoch: 2 # Don't validate before extracting features.
  fast_dev_run: false
  accumulate_grad_batches: 1
  max_epochs: 200
  min_epochs: null
  max_steps: -1
  min_steps: null
  max_time: null
  limit_train_batches: 1.0
  limit_val_batches: 1.0
  limit_test_batches: 1.0
  limit_predict_batches: 1.0
  val_check_interval: 1.0 # Don't validate before extracting features.
  log_every_n_steps: 50
  accelerator: auto # <"cpu", "gpu", "tpu", "ipu", "hpu", "auto">
  strategy: null
  sync_batchnorm: false
  precision: 32
  enable_model_summary: true
  num_sanity_val_steps: 0
  profiler: null
  benchmark: false
  deterministic: false
  reload_dataloaders_every_n_epochs: 0
  auto_lr_find: false
  replace_sampler_ddp: true
  detect_anomaly: false
  auto_scale_batch_size: false
  plugins: null
  move_metrics_to_cpu: false
  multiple_trainloader_mode: max_size_cycle

Logs

/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/config/config.py:275: UserWarning: config.project.unique_dir is set to False. This does not ensure that your results will be written in an empty directory and you may overwrite files.
  warn(
[rank: 0] Global seed set to 42
2023-07-24 15:05:07,560 - anomalib.data - INFO - Loading the datamodule
2023-07-24 15:05:07,561 - anomalib.data.utils.transform - INFO - No config file has been provided. Using default transforms.
2023-07-24 15:05:07,561 - anomalib.data.utils.transform - INFO - No config file has been provided. Using default transforms.
2023-07-24 15:05:07,562 - anomalib.models - INFO - Loading the model.
2023-07-24 15:05:07,562 - anomalib.models.components.base.anomaly_module - INFO - Initializing ReverseDistillationLightning model.
/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/torchmetrics/utilities/prints.py:36: UserWarning: Metric `PrecisionRecallCurve` will save all targets and predictions in buffer. For large datasets this may lead to large memory footprint.
  warnings.warn(*args, **kwargs)
2023-07-24 15:05:07,568 - anomalib.models.components.feature_extractors.timm - WARNING - FeatureExtractor is deprecated. Use TimmFeatureExtractor instead. Both FeatureExtractor and TimmFeatureExtractor will be removed in a future release.
2023-07-24 15:05:08,724 - timm.models.helpers - INFO - Loading pretrained weights from url (https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/wide_resnet50_racm-8234f177.pth)
2023-07-24 15:05:09,246 - anomalib.utils.loggers - INFO - Loading the experiment logger(s)
2023-07-24 15:05:09,246 - anomalib.utils.callbacks - INFO - Loading the callbacks
/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/utils/callbacks/__init__.py:142: UserWarning: Export option: None not found. Defaulting to no model export
  warnings.warn(f"Export option: {config.optimization.export_mode} not found. Defaulting to no model export")
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - GPU available: True (cuda), used: True
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - TPU available: False, using: 0 TPU cores
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - IPU available: False, using: 0 IPUs
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - HPU available: False, using: 0 HPUs
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - `Trainer(limit_train_batches=1.0)` was configured so 100% of the batches per epoch will be used..
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - `Trainer(limit_val_batches=1.0)` was configured so 100% of the batches will be used..
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - `Trainer(limit_test_batches=1.0)` was configured so 100% of the batches will be used..
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - `Trainer(limit_predict_batches=1.0)` was configured so 100% of the batches will be used..
2023-07-24 15:05:09,260 - pytorch_lightning.utilities.rank_zero - INFO - `Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..
2023-07-24 15:05:09,260 - anomalib - INFO - Training the model.
2023-07-24 15:05:09,466 - pytorch_lightning.utilities.rank_zero - INFO - You are using a CUDA device ('NVIDIA A100-SXM4-80GB MIG 1g.10gb') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
Traceback (most recent call last):
  File "/home/clusterusers/cugwu/codes/cvl/./anomalib/tools/train.py", line 79, in <module>
    train(args)
  File "/home/clusterusers/cugwu/codes/cvl/./anomalib/tools/train.py", line 64, in train
    trainer.fit(model=model, datamodule=datamodule)
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 608, in fit
    call._call_and_handle_interrupt(
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 38, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl
    self._run(model, ckpt_path=self.ckpt_path)
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1051, in _run
    self._call_setup_hook()  # allow user to setup lightning_module in accelerator environment
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1298, in _call_setup_hook
    self._call_lightning_datamodule_hook("setup", stage=fn)
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1375, in _call_lightning_datamodule_hook
    return fn(*args, **kwargs)
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/data/base/datamodule.py", line 102, in setup
    self._setup(stage)
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/data/base/datamodule.py", line 118, in _setup
    self.train_data.setup()
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/data/base/dataset.py", line 162, in setup
    self._setup()
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/data/folder.py", line 178, in _setup
    self.samples = make_folder_dataset(
  File "/home/clusterusers/cugwu/anomalib_env/lib/python3.10/site-packages/anomalib/data/folder.py", line 59, in make_folder_dataset
    assert normal_dir.is_dir(), "A folder location must be provided in normal_dir."
AssertionError: A folder location must be provided in normal_dir.

Code of Conduct

blaz-r commented 1 year ago

Hello. This could be happening since you are using relative path and your current working directory might be wrong. Are you inside root directory, where you also have datasets folder?

blaz-r commented 1 year ago

Oh, I see you installed using pip. Can you provide the code and commands that show how you call the training code?

cugwu commented 1 year ago

Hi, even I use the full path instead of the relative one I still have the problem. For calling the training code I use: python ./anomalib/tools/train.py --config ./anomalib/reverse_distillation_config.yaml

cugwu commented 1 year ago

I also tried to re-install everything and instead of using pip install anomalib[full] I cloned the repo and did: pip install e . But I still have the problem

blaz-r commented 1 year ago

Hello. Try setting the path to datasets/hazelnutand then run the program from inside the anomalib folder, so python tools/train.py...

cugwu commented 1 year ago

Thanks this last suggestion works!