search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
28.52k stars 9.29k forks source link

Error when using Albumentations augmentation in semi-supervised object detection #11227

Open NongGX opened 7 months ago

NongGX commented 7 months ago

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug Hi! I'm running into an issue when using Albumentations augmentation in semi-supervised object detection. Specifically, I get an error when using this augmentation in the strong_pipeline, but not when only using it in the sup_pipeline. Here is my config file and the full error message:

Reproduction configs/base/datasets/semi_coco_detection.py

# dataset settings
dataset_type = 'CocoDataset'
#data_root = 'data/coco/'
data_root = '/home/PycharmProjects/SoftTeacher/data/coco/'

backend_args = None

color_space = [
    [dict(type='ColorTransform')],
    [dict(type='AutoContrast')],
    [dict(type='Equalize')],
    [dict(type='Sharpness')],
    [dict(type='Posterize')],
    [dict(type='Solarize')],
    [dict(type='Color')],
    [dict(type='Contrast')],
    [dict(type='Brightness')],
]

geometric = [
    [dict(type='Rotate')],
    [dict(type='ShearX')],
    [dict(type='ShearY')],
    [dict(type='TranslateX')],
    [dict(type='TranslateY')],
]

scale = [(1333, 400), (1333, 800)]

branch_field = ['sup', 'unsup_teacher', 'unsup_student']

sup_pipeline = [
    dict(type='LoadImageFromFile', backend_args=backend_args),
    dict(type='LoadAnnotations', with_bbox=True),
    dict(type='RandomResize', scale=scale, keep_ratio=True),
    dict(type='RandomFlip', prob=0.5),
    #dict(type='RandAugment', aug_space=color_space, aug_num=1),
    dict(
        type='Albu',   # Replace with Adjustments augmentation
        transforms=[
            dict(type='ColorJitter', brightness=0.4, contrast=0.4, saturation=0.4, hue=0.1, p=0.8),
            dict(type='ToGray', p=0.2),
            dict(type='GaussianBlur', sigma_limit=(0.1, 2.0), p=0.5),
        ],
        bbox_params=dict(type='BboxParams', format='pascal_voc', label_fields=['gt_bboxes_labels','gt_ignore_flags'], min_visibility=0.0,
                         filter_lost_elements=True),
        keymap={'img': 'image', 'gt_bboxes': 'bboxes'},
        skip_img_without_anno=True
    ),
    dict(type='FilterAnnotations', min_gt_bbox_wh=(1e-2, 1e-2)),
    dict(
        type='MultiBranch',
        branch_field=branch_field,
        sup=dict(type='PackDetInputs'))
]

weak_pipeline = [
    dict(type='RandomResize', scale=scale, keep_ratio=True),
    dict(type='RandomFlip', prob=0.5),
    dict(
        type='PackDetInputs',
        meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
                   'scale_factor', 'flip', 'flip_direction',
                   'homography_matrix')),
]
strong_pipeline = [
    dict(type='RandomResize', scale=scale, keep_ratio=True),
    dict(type='RandomFlip', prob=0.5),
    #dict(
        #type='RandomOrder',
        #transforms=[
            #dict(type='RandAugment', aug_space=color_space, aug_num=1),
            #dict(type='RandAugment', aug_space=geometric, aug_num=1),
        #]),
     dict(
        type='Albu',          # Replace with Adjustments augmentation
        transforms=[
            dict(type='ColorJitter', brightness=0.4, contrast=0.4, saturation=0.4, hue=0.1, p=0.8),
            dict(type='ToGray', p=0.2),
            dict(type='GaussianBlur', sigma_limit=(0.1, 2.0), p=0.5),
        ],
        bbox_params=dict(type='BboxParams', format='pascal_voc', label_fields=['gt_bboxes_labels','gt_ignore_flags'], min_visibility=0.0,
                         filter_lost_elements=True),
        keymap={'img': 'image', 'gt_bboxes': 'bboxes'},
        skip_img_without_anno=True
    ),
    dict(type='RandomErasing', n_patches=(1, 5), ratio=(0, 0.2)),
    dict(type='FilterAnnotations', min_gt_bbox_wh=(1e-2, 1e-2)),
    dict(
        type='PackDetInputs',
        meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
                   'scale_factor', 'flip', 'flip_direction',
                   'homography_matrix')),
]

unsup_pipeline = [
    dict(type='LoadImageFromFile', backend_args=backend_args),
    dict(type='LoadEmptyAnnotations'),
    dict(
        type='MultiBranch',
        branch_field=branch_field,
        unsup_teacher=weak_pipeline,
        unsup_student=strong_pipeline,
    )
]

test_pipeline = [
    dict(type='LoadImageFromFile', backend_args=backend_args),
    dict(type='Resize', scale=(1333, 800), keep_ratio=True),
    dict(
        type='PackDetInputs',
        meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',
                   'scale_factor'))
]

batch_size = 5
num_workers = 5

labeled_dataset = dict(
    type=dataset_type,
    data_root=data_root,
    ann_file='annotations/instances_train2017.json',
    data_prefix=dict(img='train2017/'),
    filter_cfg=dict(filter_empty_gt=True, min_size=32),
    pipeline=sup_pipeline,
    backend_args=backend_args)

unlabeled_dataset = dict(
    type=dataset_type,
    data_root=data_root,
    ann_file='annotations/instances_unlabeled2017.json',
    data_prefix=dict(img='unlabeled2017/'),
    filter_cfg=dict(filter_empty_gt=False),
    pipeline=unsup_pipeline,
    backend_args=backend_args)

train_dataloader = dict(
    batch_size=batch_size,
    num_workers=num_workers,
    persistent_workers=True,
    sampler=dict(
        type='GroupMultiSourceSampler',
        batch_size=batch_size,
        source_ratio=[1, 4]),
    dataset=dict(
        type='ConcatDataset', datasets=[labeled_dataset, unlabeled_dataset]))

val_dataloader = dict(
    batch_size=1,
    num_workers=2,
    persistent_workers=True,
    drop_last=False,
    sampler=dict(type='DefaultSampler', shuffle=False),
    dataset=dict(
        type=dataset_type,
        data_root=data_root,
        ann_file='annotations/instances_val2017.json',
        data_prefix=dict(img='val2017/'),
        test_mode=True,
        pipeline=test_pipeline,
        backend_args=backend_args))

test_dataloader = val_dataloader

val_evaluator = dict(
    type='CocoMetric',
    ann_file=data_root + 'annotations/instances_val2017.json',
    metric='bbox',
    format_only=False,
    backend_args=backend_args)
test_evaluator = val_evaluator

Error traceback

11/29 21:42:22 - mmengine - INFO - load model from: open-mmlab://detectron2/resnet50_caffe 11/29 21:42:22 - mmengine - INFO - Loads checkpoint by openmmlab backend from path: open-mmlab://detectron2/resnet50_caffe 11/29 21:42:22 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io 11/29 21:42:22 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future. 11/29 21:42:22 - mmengine - INFO - Checkpoints will be saved to /home/nongxx/PycharmProjects/mmdetection/tools/work_dirs/soft-teacher-coco-idea. Traceback (most recent call last): File "/home/PycharmProjects/mmdetection/tools/train.py", line 133, in main() File "/home/PycharmProjects/mmdetection/tools/train.py", line 129, in main runner.train() File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1706, in train model = self.train_loop.run() # type: ignore File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/mmengine/runner/loops.py", line 277, in run data_batch = next(self.dataloader_iterator) File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/mmengine/runner/loops.py", line 164, in next data = next(self._iterator) File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 628, in next data = self._next_data() File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data return self._process_data(data) File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data data.reraise() File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/torch/_utils.py", line 543, in reraise raise exception Exception: Caught Exception in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index) File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 58, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/mmengine/dataset/dataset_wrapper.py", line 159, in getitem return self.datasets[dataset_idx][sample_idx] File "/home/anaconda3/envs/pytorch1.13/lib/python3.10/site-packages/mmengine/dataset/base_dataset.py", line 421, in getitem raise Exception(f'Cannot find valid image after {self.max_refetch}! ' Exception: Cannot find valid image after 1000! Please check your image path and pipeline

The error message above shows that I need to check the path, but I can run the official data augmentation code normally (the commented part)，so please let me know if any other details would be helpful in this problem.

jbesq-ml commented 5 months ago

Did you resolve this issue?