search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.49k stars 9.44k forks source link

Albumentations error #11925

Open pedrozamboni opened 2 months ago

pedrozamboni commented 2 months ago

Hey, Im having some issues while trying to use Albumentations direct on my config file. I always get this error:

File "tools/train.py", line 247, in main() File "tools/train.py", line 236, in main train_detector( File "/home/pedro/Documents/Documents/mmdetection-2.28.2/mmdet/apis/train.py", line 246, in train_detector runner.run(data_loaders, cfg.workflow) File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 136, in run epoch_runner(data_loaders[i], kwargs) File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 49, in train for i, data_batch in enumerate(self.data_loader): File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 634, in next data = self._next_data() File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1346, in _next_data return self._process_data(data) File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1372, in _process_data data.reraise() File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise raise exception ValueError: Caught ValueError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/pedro/Documents/Documents/mmdetection-2.28.2/mmdet/datasets/custom.py", line 220, in getitem data = self.prepare_train_img(idx) File "/home/pedro/Documents/Documents/mmdetection-2.28.2/mmdet/datasets/custom.py", line 243, in prepare_train_img return self.pipeline(results) File "/home/pedro/Documents/Documents/mmdetection-2.28.2/mmdet/datasets/pipelines/compose.py", line 41, in call data = t(data) File "/home/pedro/Documents/Documents/mmdetection-2.28.2/mmdet/datasets/pipelines/transforms.py", line 1487, in call results = self.aug(results) File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/albumentations/core/composition.py", line 304, in call self.preprocess(data) File "/home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/albumentations/core/composition.py", line 331, in preprocess raise ValueError(msg) ValueError: Key img_info is not in available keys.

I`m using a custom dataset and a custom loading functions to open npy files with 5 bands.

This is my loading func:

class LoadImageFromNumpy(object): """Load an image from file.

Required keys are "img_prefix" and "img_info" (a dict that must contain the
key "filename"). Added or updated keys are "filename", "img", "img_shape",
"ori_shape" (same as `img_shape`), "pad_shape" (same as `img_shape`),
"scale_factor" (1.0) and "img_norm_cfg" (means=0 and stds=1).

Args:
    to_float32 (bool): Whether to convert the loaded image to a float32
        numpy array. If set to False, the loaded image is an uint8 array.
        Defaults to False.
    color_type (str): The flag argument for :func:`mmcv.imfrombytes`.
        Defaults to 'color'.
    file_client_args (dict): Arguments to instantiate a FileClient.
        See :class:`mmcv.fileio.FileClient` for details.
        Defaults to ``dict(backend='disk')``.
    imdecode_backend (str): Backend for :func:`mmcv.imdecode`. Default:
        'cv2'
"""

def __init__(self,
             to_float32=False,
             color_type='color',
             file_client_args=dict(backend='disk'),
             imdecode_backend='numpy',
             bands=[0,1,2],
             ):
    self.to_float32 = to_float32
    self.color_type = color_type
    self.file_client_args = file_client_args.copy()
    self.file_client = None
    self.imdecode_backend = imdecode_backend
    self.bands = bands

def __call__(self, results):
    """Call functions to load image and get image meta information.

    Args:
        results (dict): Result dict from :obj:`mmseg.CustomDataset`.

    Returns:
        dict: The dict contains loaded image and meta information.
    """

    if self.file_client is None:
        self.file_client = mmcv.FileClient(**self.file_client_args)

    if results.get('img_prefix') is not None:
        filename = osp.join(results['img_prefix'],
                            results['img_info']['filename'])
    else:
        filename = results['img_info']['filename']

    #img_bytes = np.load(filename.replace('.png','.npy'))
    #print(filename.replace('.png','.npy'))
    img_bytes = np.load(filename)
    img = img_bytes[:,:,self.bands]
    if self.to_float32:
        img = img.astype(np.float32)

    results['filename'] = filename
    results['ori_filename'] = results['img_info']['filename']
    results['img'] = img
    results['img_shape'] = img.shape
    results['ori_shape'] = img.shape
    # Set initial values for default meta_keys
    results['pad_shape'] = img.shape
    #results['scale_factor'] = 1.0
    num_channels = 1 if len(img.shape) < 3 else img.shape[2]

    return results

Environment /home/pedro/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/init.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details. warnings.warn( fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). sys.platform: linux Python: 3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:47:35) [GCC 12.3.0] CUDA available: True GPU 0: NVIDIA GeForce RTX 3090 Ti CUDA_HOME: /usr NVCC: Cuda compilation tools, release 11.5, V11.5.119 GCC: gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 PyTorch: 2.0.0+cu118 PyTorch compiling details: PyTorch built with:

TorchVision: 0.15.1+cu118 OpenCV: 4.10.0 MMCV: 1.7.2 MMCV Compiler: GCC 9.3 MMCV CUDA Compiler: 11.8 MMDetection: 2.28.2+

Sologala commented 2 days ago

same question