search

open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.6k stars 9.47k forks source link

AttributeError: 'ActionDataSample' object has no attribute 'gt_labels' #11505

Open TonsonP opened 8 months ago

TonsonP commented 8 months ago

Thanks for your error report and we appreciate it a lot.

Describe the bug I encounter the error "AttributeError: 'ActionDataSample' object has no attribute 'gt_labels'" when I try to train actionclip model with nturgb_d dataset.

Reproduction

  1. What command or script did you run?
mim train mmaction configs/actionclip_vit-base-p16-res224-clip-pre_g8xb16_1x1x8_k400-rgb_nturgb_d.py

  1. Did you make any modifications on the code or config? Did you understand what you have modified? I modified the path of label file, data_root, data_root_val and the path of annotation. Modified from "actionclip_vit-base-p16-res224-clip-pre_g8xb16_1x1x8_k400-rgb.py" file. I also modified the batch size of each loader (train, val test) as I tried to train on single gpu.

  2. What dataset did you use? nturgb_d dataset.

Environment

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here. 
    
sys.platform: linux
Python: 3.8.18 | packaged by conda-forge | (default, Dec 23 2023, 17:21:28) [GCC 12.3.0]
CUDA available: True
MUSA available: False
numpy_random_seed: 2147483648
GPU 0: NVIDIA GeForce RTX 3060 Ti
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.8, V11.8.89
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
PyTorch: 2.0.1+cu117
PyTorch compiling details: PyTorch built with:
- GCC 9.3
- C++ Version: 201703
- Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.7
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
- CuDNN 8.5
- Magma 2.6.1
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.15.2+cu117 OpenCV: 4.9.0 MMEngine: 0.10.3 MMAction2: 1.2.0+4d6c934 MMCV: 2.1.0 MMDetection: 3.2.0 MMPose: 1.3.1

2. Additional info
- I used the conda environment, followed the best practice in this guide (https://mmaction2.readthedocs.io/en/latest/get_started/installation.html)
-  For the $PYTHONPATH, I just follow the instruction from (https://github.com/open-mmlab/mmaction2/tree/main/projects/actionclip)

Here is the full config file that I tried to used.

```python
custom_imports = dict(imports='models')

num_segs = 8

model = dict(
    type='ActionClip',
    clip_arch='ViT-B/16',
    num_adapter_segs=num_segs,
    num_adapter_layers=6,
    to_float32=True,
    labels_or_label_file='configs/nturgb_d_label.txt',
    data_preprocessor=dict(
        type='ActionDataPreprocessor',
        mean=[122.771, 116.746, 104.093],
        std=[68.500, 66.632, 70.323],
        format_shape='NCHW'))

dataset_type = 'VideoDataset'
data_root = "/ntu_rgbd_dataset/nturgb+d_rgb"
data_root_val = "/ntu_rgbd_dataset/nturgb+d_rgb"
ann_file_train = "/self_anno/nturgb_d_train_set.txt"
ann_file_val = "/self_anno/nturgb_d_val_set.txt"
ann_file_test = "/self_anno/nturgb_d_test_set.txt"

file_client_args = dict(io_backend='disk')
# file_client_args = dict(
#     io_backend='petrel',
#     path_mapping=dict(
#         {'data/nturgb_d/': '/home/tonson/Documents/ntu_rgbd_dataset/nturgb+d_rgb'}))

train_pipeline = [
    dict(type='DecordInit', **file_client_args),
    dict(
        type='SampleFrames', clip_len=1, frame_interval=1, num_clips=num_segs),
    dict(type='DecordDecode'),
    dict(type='Resize', scale=(-1, 256)),
    dict(type='RandomResizedCrop'),
    dict(
        type='MultiScaleCrop',
        input_size=224,
        scales=(1, .875, .75, .66),
        random_crop=False,
        num_fixed_crops=13,
        max_wh_scale_gap=1),
    dict(type='Resize', scale=(224, 224), keep_ratio=False),
    dict(type='Flip', flip_ratio=0.5),
    dict(type='FormatShape', input_format='NCHW'),
    dict(type='PackActionInputs')

]

val_pipeline = [
    dict(type='DecordInit', **file_client_args),
    dict(
        type='SampleFrames',
        clip_len=1,
        frame_interval=1,
        num_clips=num_segs,
        test_mode=True),
    dict(type='DecordDecode'),
    dict(type='Resize', scale=(-1, 256)),
    dict(type='CenterCrop', crop_size=224),
    dict(type='FormatShape', input_format='NCHW'),
    dict(type='PackActionInputs')
]

test_pipeline = val_pipeline

# original batch_size is 16

train_dataloader = dict(
    batch_size=4,
    num_workers=16,
    persistent_workers=True,
    sampler=dict(type='DefaultSampler', shuffle=True),
    dataset=dict(
        type=dataset_type,
        ann_file=ann_file_train,
        data_prefix=dict(video=data_root),
        pipeline=train_pipeline))
val_dataloader = dict(
    batch_size=8,
    num_workers=16,
    persistent_workers=True,
    sampler=dict(type='DefaultSampler', shuffle=False),
    dataset=dict(
        type=dataset_type,
        ann_file=ann_file_val,
        data_prefix=dict(video=data_root_val),
        pipeline=val_pipeline,
        test_mode=True))
test_dataloader = dict(
    batch_size=1,
    num_workers=16,
    persistent_workers=True,
    sampler=dict(type='DefaultSampler', shuffle=False),
    dataset=dict(
        type=dataset_type,
        ann_file=ann_file_test,
        data_prefix=dict(video=data_root_val),
        pipeline=test_pipeline,
        test_mode=True))

val_evaluator = dict(type='AccMetric')
test_evaluator = val_evaluator

train_cfg = dict(
    type='EpochBasedTrainLoop', max_epochs=50, val_begin=1, val_interval=1)
val_cfg = dict(type='ValLoop')
test_cfg = dict(type='TestLoop')

optim_wrapper = dict(
    optimizer=dict(
        type='AdamW', lr=5e-6, betas=(0.9, 0.98), eps=1e-08, weight_decay=0.2),
    paramwise_cfg=dict(custom_keys=dict(adapter=dict(lr_mult=10))))

param_scheduler = [
    dict(
        type='LinearLR',
        start_factor=0.01,
        by_epoch=True,
        begin=0,
        end=5,
        convert_to_iter_based=True),
    dict(
        type='CosineAnnealingLR',
        T_max=45,
        eta_min=0,
        by_epoch=True,
        begin=5,
        end=50,
        convert_to_iter_based=True)
]

# Default setting for scaling LR automatically
#   - `enable` means enable scaling LR automatically
#       or not by default.
#   - `base_batch_size` = (8 GPUs) x (16 samples per GPU).
auto_scale_lr = dict(enable=False, base_batch_size=128)

default_scope = 'mmaction'

default_hooks = dict(
    runtime_info=dict(type='RuntimeInfoHook'),
    timer=dict(type='IterTimerHook'),
    logger=dict(type='LoggerHook', interval=100, ignore_last=False),
    param_scheduler=dict(type='ParamSchedulerHook'),
    checkpoint=dict(
        type='CheckpointHook', interval=1, save_best='auto', max_keep_ckpts=5),
    sampler_seed=dict(type='DistSamplerSeedHook'),
    sync_buffers=dict(type='SyncBuffersHook'))

env_cfg = dict(
    cudnn_benchmark=False,
    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
    dist_cfg=dict(backend='nccl'))

log_processor = dict(type='LogProcessor', window_size=20, by_epoch=True)

vis_backends = [dict(type='LocalVisBackend')]
visualizer = dict(type='ActionVisualizer', vis_backends=vis_backends)

log_level = 'INFO'
load_from = None
resume = False

Error traceback

Traceback (most recent call last):
  File "/home/tonson/Documents/mmaction2/mmaction/.mim/tools/train.py", line 143, in <module>
    main()
  File "/home/tonson/Documents/mmaction2/mmaction/.mim/tools/train.py", line 139, in main
    runner.train()
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1777, in train
    model = self.train_loop.run()  # type: ignore
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/runner/loops.py", line 96, in run
    self.run_epoch()
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/runner/loops.py", line 112, in run_epoch
    self.run_iter(idx, data_batch)
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/runner/loops.py", line 128, in run_iter
    outputs = self.runner.model.train_step(
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 114, in train_step
    losses = self._run_forward(data, mode='loss')  # type: ignore
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 361, in _run_forward
    results = self(**data, mode=mode)
  File "/home/tonson/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/tonson/Documents/mmaction2/projects/actionclip/models/actionclip.py", line 146, in forward
    real_labels = [x.gt_labels.item.item() for x in data_samples]
  File "/home/tonson/Documents/mmaction2/projects/actionclip/models/actionclip.py", line 146, in <listcomp>
    real_labels = [x.gt_labels.item.item() for x in data_samples]
AttributeError: 'ActionDataSample' object has no attribute 'gt_labels'
Traceback (most recent call last):
  File "/home/tonson/miniconda3/envs/openmmlab/bin/mim", line 8, in <module>
    sys.exit(cli())
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mim/commands/train.py", line 100, in cli
    is_success, msg = train(
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/site-packages/mim/commands/train.py", line 261, in train
    ret = subprocess.check_call(
  File "/home/tonson/miniconda3/envs/openmmlab/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/tonson/miniconda3/envs/openmmlab/bin/python', '/home/tonson/Documents/mmaction2/mmaction/.mim/tools/train.py', 'configs/actionclip_vit-base-p16-res224-clip-pre_g8xb16_1x1x8_k400-rgb_nturgb_d.py', '--launcher', 'none']' returned non-zero exit status 1.

Any suggestion is appreciate!

TonsonP commented 8 months ago

Turn out there is some issue on this line. https://github.com/open-mmlab/mmaction2/blob/4d6c93474730cad2f25e51109adcf96824efc7a3/projects/actionclip/models/actionclip.py#L146

I can fix it by replacing it with real_labels = [x.gt_label.item() -1 for x in data_samples]