search

open-mmlab / mmpretrain

OpenMMLab Pre-training Toolbox and Benchmark
https://mmpretrain.readthedocs.io/en/latest/
Apache License 2.0
3.38k stars 1.05k forks source link

[Bug] AttributeError: 'NoneType' object has no attribute 'shape' #1427

Open marouaneamz opened 1 year ago

marouaneamz commented 1 year ago

Branch

1.x branch (1.0.0rc2 or other 1.x version)

Describe the bug

I have been working on the subject of multi-tasking for a long time to improve it, I train the model on the same data but today I encountered a problem for the shape of images I have checked that the image exists and the path is correct and I have not changed in the configuration but the img in result is NONE {'img_path': '.././data/indoor\buffet/Buffet_Lettuce_gif.jpg', 'gt_label': {'Indoor': 9}, 'img': None}

Original Traceback (most recent call last):
  File "C:\Users\FR00CSS0000000040678\Anaconda3\envs\myenv\lib\site-packages\torch\utils\data\_utils\worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "C:\Users\FR00CSS0000000040678\Anaconda3\envs\myenv\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\FR00CSS0000000040678\Anaconda3\envs\myenv\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "c:\users\fr00css0000000040678\desktop\openmmlab\mmclassification\mmcls\datasets\multi_task.py", line 316, in __getitem__
    return self.prepare_data(idx)
  File "c:\users\fr00css0000000040678\desktop\openmmlab\mmclassification\mmcls\datasets\multi_task.py", line 295, in prepare_data
    return self.pipeline(results)
  File "C:\Users\FR00CSS0000000040678\Anaconda3\envs\myenv\lib\site-packages\mmcv\transforms\base.py", line 12, in __call__
    return self.transform(results)
  File "C:\Users\FR00CSS0000000040678\Anaconda3\envs\myenv\lib\site-packages\mmcv\transforms\wrappers.py", line 87, in transform
    results = t(results)  # type: ignore
  File "C:\Users\FR00CSS0000000040678\Anaconda3\envs\myenv\lib\site-packages\mmcv\transforms\base.py", line 12, in __call__
    return self.transform(results)
  File "C:\Users\FR00CSS0000000040678\Anaconda3\envs\myenv\lib\site-packages\mmcv\transforms\loading.py", line 112, in transform
    results['img_shape'] = img.shape[:2]
AttributeError: 'NoneType' object has no attribute 'shape'

Environment

{'sys.platform': 'win32', 'Python': '3.8.16 (default, Jan 17 2023, 22:25:28) [MSC v.1916 64 bit ' '(AMD64)]', 'CUDA available': True, 'numpy_random_seed': 2147483648, 'GPU 0': 'NVIDIA GeForce RTX 2060 with Max-Q Design', 'CUDA_HOME': None, 'GCC': 'n/a', 'PyTorch': '1.10.1', 'PyTorch compiling details': 'PyTorch built with:\n' ' - C++ Version: 199711\n' ' - MSVC 192829337\n' ' - Intel(R) Math Kernel Library Version ' '2020.0.2 Product Build 20200624 for Intel(R) 64 ' 'architecture applications\n' ' - Intel(R) MKL-DNN v2.2.3 (Git Hash ' '7336ca9f055cf1bfa13efb658fe15dc9b41f0740)\n' ' - OpenMP 2019\n' ' - LAPACK is enabled (usually provided by ' 'MKL)\n' ' - CPU capability usage: AVX2\n' ' - CUDA Runtime 11.3\n' ' - NVCC architecture flags: ' '-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37\n' ' - CuDNN 8.2\n' ' - Magma 2.5.4\n' ' - Build settings: BLAS_INFO=mkl, ' 'BUILD_TYPE=Release, CUDA_VERSION=11.3, ' 'CUDNN_VERSION=8.2.0, ' 'CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, ' 'CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w ' '/bigobj -DUSE_PTHREADPOOL -openmp:experimental ' '-IC:/cb/pytorch_1000000000000/work/mkl/include ' '-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI ' '-DUSE_FBGEMM -DUSE_XNNPACK ' '-DSYMBOLICATE_MOBILE_DEBUG_HANDLE ' '-DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, ' 'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, ' 'PERF_WITH_AVX512=1, TORCH_VERSION=1.10.1, ' 'USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, ' 'USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, ' 'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, ' 'USE_NNPACK=OFF, USE_OPENMP=ON, \n', 'TorchVision': '0.11.2', 'OpenCV': '4.7.0', 'MMEngine': '0.5.0', 'MMClassification': '1.0.0rc5+b47cd59'} (myenv)

Other information

No response

Ezra-Yu commented 1 year ago 
'img_path': '.././data/indoor\buffet/Buffet_Lettuce_gif.jpg', 'gt_label': {'Indoor': 9}, 'img': None}

It seems that it failed to LoadImageFromFile, maybe the path is wrong.

marouaneamz commented 1 year ago 
'img_path': '.././data/indoor\buffet/Buffet_Lettuce_gif.jpg', 'gt_label': {'Indoor': 9}, 'img': None}

It seems that it failed to LoadImageFromFile, maybe the path is wrong.

I checked the path it works well, when I raise imgbytes()in mmcv loading.py I have results the problem is to transform from bytes to array

Ezra-Yu commented 1 year ago

One solution is to place the reading of images in the dataset class， for example Minist.

VJoer commented 1 year ago

I also get seam error in CustomDataset dataload. I following this Documentation set config, replace data_root with my own path

_base_ = [
    '../_base_/models/mae_vit-base-p16.py',
    '../_base_/datasets/imagenet_bs512_mae.py',
    '../_base_/default_runtime.py',
]

train_dataloader = dict(
    dataset=dict(
        type='CustomDataset',
        data_root='/world/data-c9/batch/classification',
        ann_file='',       # 我们假定使用子文件夹格式，因此需要将标注文件置空
        data_prefix='',    # 使用 `data_root` 路径下所有数据
        with_label=True,
    )
)

# optimizer wrapper
optim_wrapper = dict(
    type='AmpOptimWrapper',
    loss_scale='dynamic',
    optimizer=dict(
        type='AdamW',
        lr=1.5e-4 * 4096 / 256,
        betas=(0.9, 0.95),
        weight_decay=0.05),
    paramwise_cfg=dict(
        custom_keys={
            'ln': dict(decay_mult=0.0),
            'bias': dict(decay_mult=0.0),
            'pos_embed': dict(decay_mult=0.),
            'mask_token': dict(decay_mult=0.),
            'cls_token': dict(decay_mult=0.)
        }))

# learning rate scheduler
param_scheduler = [
    dict(
        type='LinearLR',
        start_factor=0.0001,
        by_epoch=True,
        begin=0,
        end=40,
        convert_to_iter_based=True),
    dict(
        type='CosineAnnealingLR',
        T_max=260,
        by_epoch=True,
        begin=40,
        end=300,
        convert_to_iter_based=True)
]

# runtime settings
train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=300)
default_hooks = dict(
    # only keeps the latest 3 checkpoints
    checkpoint=dict(type='CheckpointHook', interval=1, max_keep_ckpts=3))

randomness = dict(seed=0, diff_rank_seed=True)

# auto resume
resume = True

# NOTE: `auto_scale_lr` is for automatically scaling LR
# based on the actual training batch size.
auto_scale_lr = dict(base_batch_size=4096)

get error 

Traceback (most recent call last):
  File "tools/train.py", line 159, in <module>
    main()
  File "tools/train.py", line 155, in main
    runner.train()
  File "/usr/local/lib/python3.8/dist-packages/mmengine/runner/runner.py", line 1686, in train
    model = self.train_loop.run()  # type: ignore
  File "/usr/local/lib/python3.8/dist-packages/mmengine/runner/loops.py", line 90, in run
    self.run_epoch()
  File "/usr/local/lib/python3.8/dist-packages/mmengine/runner/loops.py", line 105, in run_epoch
    for idx, data_batch in enumerate(self.dataloader):
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 628, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 1333, in _next_data
    return self._process_data(data)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 1359, in _process_data
    data.reraise()
  File "/usr/local/lib/python3.8/dist-packages/torch/_utils.py", line 543, in reraise
    raise exception
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.8/dist-packages/mmengine/dataset/base_dataset.py", line 408, in __getitem__
    data = self.prepare_data(idx)
  File "/usr/local/lib/python3.8/dist-packages/mmengine/dataset/base_dataset.py", line 789, in prepare_data
    return self.pipeline(data_info)
  File "/usr/local/lib/python3.8/dist-packages/mmengine/dataset/base_dataset.py", line 58, in __call__
    data = t(data)
  File "/usr/local/lib/python3.8/dist-packages/mmcv/transforms/base.py", line 12, in __call__
    return self.transform(results)
  File "/usr/local/lib/python3.8/dist-packages/mmcv/transforms/loading.py", line 112, in transform
    results['img_shape'] = img.shape[:2]
AttributeError: 'NoneType' object has no attribute 'shape'

my environment

05/08 16:49:06 - mmengine - INFO - 

05/08 16:49:06 - mmengine - INFO - **********Environmental information**********
05/08 16:49:08 - mmengine - INFO - sys.platform: linux
05/08 16:49:08 - mmengine - INFO - Python: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
05/08 16:49:08 - mmengine - INFO - CUDA available: True
05/08 16:49:08 - mmengine - INFO - numpy_random_seed: 2147483648
05/08 16:49:08 - mmengine - INFO - GPU 0,1: NVIDIA GeForce GTX 1080 Ti
05/08 16:49:08 - mmengine - INFO - CUDA_HOME: /usr/local/cuda
05/08 16:49:08 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.7, V11.7.99
05/08 16:49:08 - mmengine - INFO - GCC: x86_64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
05/08 16:49:08 - mmengine - INFO - PyTorch: 1.13.0+cu117
05/08 16:49:08 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.7
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.5
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

05/08 16:49:08 - mmengine - INFO - TorchVision: 0.14.0+cu117
05/08 16:49:08 - mmengine - INFO - OpenCV: 4.7.0
05/08 16:49:08 - mmengine - INFO - MMEngine: 0.6.0
05/08 16:49:08 - mmengine - INFO - MMCV: 2.0.0rc4
05/08 16:49:08 - mmengine - INFO - MMCV Compiler: GCC 9.3
05/08 16:49:08 - mmengine - INFO - MMCV CUDA Compiler: 11.7
05/08 16:49:08 - mmengine - INFO - MMDeploy: 1.0.0rc3+84a289f
05/08 16:49:08 - mmengine - INFO - 

05/08 16:49:08 - mmengine - INFO - **********Backend information**********
05/08 16:49:08 - mmengine - INFO - tensorrt:    8.4.2.4
05/08 16:49:08 - mmengine - INFO - tensorrt custom ops: Available
05/08 16:49:08 - mmengine - INFO - ONNXRuntime: None
05/08 16:49:08 - mmengine - INFO - ONNXRuntime-gpu:     1.12.0
05/08 16:49:08 - mmengine - INFO - ONNXRuntime custom ops:      Available
05/08 16:49:08 - mmengine - INFO - pplnn:       None
05/08 16:49:08 - mmengine - INFO - ncnn:        None
05/08 16:49:08 - mmengine - INFO - snpe:        None
05/08 16:49:08 - mmengine - INFO - openvino:    None
05/08 16:49:08 - mmengine - INFO - torchscript: 1.13.0+cu117
05/08 16:49:08 - mmengine - INFO - torchscript custom ops:      NotAvailable
05/08 16:49:08 - mmengine - INFO - rknn-toolkit:        None
05/08 16:49:08 - mmengine - INFO - rknn-toolkit2:       None
05/08 16:49:08 - mmengine - INFO - ascend:      None
05/08 16:49:08 - mmengine - INFO - coreml:      None
05/08 16:49:08 - mmengine - INFO - tvm: None
05/08 16:49:08 - mmengine - INFO - vacc:        None
05/08 16:49:08 - mmengine - INFO - 

05/08 16:49:08 - mmengine - INFO - **********Codebase information**********
05/08 16:49:08 - mmengine - INFO - mmdet:       3.0.0rc6
05/08 16:49:08 - mmengine - INFO - mmseg:       None
05/08 16:49:08 - mmengine - INFO - mmcls:       None
05/08 16:49:08 - mmengine - INFO - mmocr:       None
05/08 16:49:08 - mmengine - INFO - mmedit:      None
05/08 16:49:08 - mmengine - INFO - mmdet3d:     None
05/08 16:49:08 - mmengine - INFO - mmpose:      None
05/08 16:49:08 - mmengine - INFO - mmrotate:    None
05/08 16:49:08 - mmengine - INFO - mmaction:    None
Ezra-Yu commented 1 year ago

@VJoer 

    results['img_shape'] = img.shape[:2]
AttributeError: 'NoneType' object has no attribute 'shape'

It is because that loading images from your dataset failed. Please check your dataset by using this t ools https://mmpretrain.readthedocs.io/en/latest/useful_tools/dataset_visualization.html

VJoer commented 1 year ago

It is because that loading images from your dataset failed. Please check your dataset by using this t ools https://mmpretrain.readthedocs.io/en/latest/useful_tools/dataset_visualization.html

@Ezra-Yu I'm sorry it doesn't work because of the same 

  File "/usr/local/lib/python3.8/dist-packages/mmcv/transforms/loading.py", line 112, in transform
    results['img_shape'] = img.shape[:2]
AttributeError: 'NoneType' object has no attribute 'shape'

my cmd is python tools/visualization/browse_dataset.py configs/mae/test.py -o /world/data-c9/temp/test/0 -n 5

Ezra-Yu commented 1 year ago

Check your dataset if there is a '.gif' picture.

VJoer commented 1 year ago

I checked. There are no '.gif' images

Ezra-Yu commented 1 year ago

To debug, https://github.com/open-mmlab/mmcv/blob/89a264527e3dc9c5eebed6195faa709d446c7a9c/mmcv/transforms/loading.py#L111

add a sentence before this line:

assert img is not None, f'{results}'

Then, you will get the error image path

VJoer commented 1 year ago 
  File "/usr/local/lib/python3.8/dist-packages/mmcv/transforms/loading.py", line 111, in transform
    assert img is not None, f'{results}'
AssertionError: {'img_path': '/world/data-c9/batch/classification/0/1676361315487_0.13458922549623908.jpg', 'sample_idx': 0}
root@gz-cs-gpu-3-107:/world/data-c9/jiangxiaowei/mmlab/mmpretrain# ll /world/data-c9/batch/classification/0/1676361315487_0.13458922549623908.jpg
-rwxr-xr-x 1 nobody nogroup 86976 May  8 15:54 /world/data-c9/batch/classification/0/1676361315487_0.13458922549623908.jpg*

It looks like not read image

Ezra-Yu commented 1 year ago

something wrong with loading this image world/data-c9/jiangxiaowei/mmlab/mmpretrain# ll /world/data-c9/batch/classification/0/1676361315487_0.13458922549623908.jpg

VJoer commented 1 year ago

To debug, https://github.com/open-mmlab/mmcv/blob/89a264527e3dc9c5eebed6195faa709d446c7a9c/mmcv/image/io.py#L285 add a sentence after this line: assert img is not None, f'{img_np, flag}'

  File "/usr/local/lib/python3.8/dist-packages/mmcv/image/io.py", line 287, in imfrombytes
    assert img is not None, f'{img_np, flag}'
AssertionError: (array([71, 73, 70, ...,  0,  0, 59], dtype=uint8), 1)

it saem like cv2.imdecode() this function is not working correctly

Ezra-Yu commented 1 year ago

@VJoer You can try to add backend='pillow' in train_dataloader.dataset.pipelines as following:

some images cannot be decoded by opencv by PIL can decode them well.

train_pipeline = [
    dict(type='LoadImageFromFile', backend='pillow'),
    dict(type='RandomResizedCrop', scale=224, backend='pillow'),
    dict(type='RandomFlip', prob=0.5, direction='horizontal'),
    dict(type='PackInputs'),
]
VJoer commented 1 year ago

it work for me, in my test it not backend but imdecode_backend

train_pipeline = [
    dict(type='LoadImageFromFile', imdecode_backend='pillow'),
    dict(type='RandomResizedCrop', scale=224, backend='pillow'),
    dict(type='RandomFlip', prob=0.5, direction='horizontal'),
    dict(type='PackInputs'),
]