VinceChin
commented
7 months ago Hi andrew. I run across exactly the same problem when I tried to import my own frame based dataset. Could you tell me whether you know how to solve it?
Open andrewis88 opened 9 months ago
VinceChin
commented
7 months ago Hi andrew. I run across exactly the same problem when I tried to import my own frame based dataset. Could you tell me whether you know how to solve it?
WANGCHAO1996
commented
7 months ago aset. Could you tell me whether you know how to solve it?
你好 请问这个问题解决了吗？
azhen01
commented
4 months ago aset. Could you tell me whether you know how to solve it?
你好 请问这个问题解决了吗？
你好，请问这个问题解决了吗？
Y-Sisyphus
commented
4 months ago @andrewis88 You should change the pipeline correspondingly: DecordInit is used to load video, you should delete it. Besides, you should change DecordDecode into RawFrameDecode.
Branch
main branch (1.x version, such as
v1.0.0, ordev-1.xbranch)Prerequisite
Environment
System environment: sys.platform: linux Python: 3.8.18 (default, Sep 11 2023, 13:40:15) [GCC 11.2.0] CUDA available: True numpy_random_seed: 1335952062 GPU 0: NVIDIA GeForce GTX 1050 Ti CUDA_HOME: //usr/local/cuda-11.2 NVCC: Cuda compilation tools, release 11.2, V11.2.67 GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 PyTorch: 1.8.0 PyTorch compiling details: PyTorch built with:
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
TorchVision: 0.9.0 OpenCV: 4.7.0 MMEngine: 0.8.5
Describe the bug
I use configs/recognition/slowfast/slowfast_r50_8xb8-4x16x1-256e_kinetics400-rgb.py to run hmdb51,and this is my set below:
dataset_type = 'RawframeDataset' data_root = 'data/hmdb51/rawframes'
data_root_val = 'data/hmdb51/rawframes'
ann_file_train = 'data/hmdb51/hmdb51_train_split_1_rawframes.tx ann_file_val = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
ann_file_test = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
the folder structure will look like: ├── data │ ├── hmdb51 │ │ ├── hmdb51_{train,val}split{1,2,3}rawframes.txt │ │ ├── hmdb51{train,val}split{1,2,3}_videos.txt │ │ ├── annotations │ │ ├── videos │ │ │ ├── brush_hair │ │ │ │ ├── April_09_brush_hair_u_nm_np1_ba_goo_0.avi
│ │ │ ├── wave │ │ │ │ ├── 20060723sfjffbartsinger_wave_f_cm_np1_ba_med_0.avi │ │ ├── rawframes │ │ │ ├── brush_hair │ │ │ │ ├── April_09_brush_hair_u_nm_np1_ba_goo_0 │ │ │ │ │ ├── img_00001.jpg │ │ │ │ │ ├── img_00002.jpg │ │ │ │ │ ├── ... │ │ │ │ │ ├── flow_x_00001.jpg │ │ │ │ │ ├── flow_x_00002.jpg │ │ │ │ │ ├── ...
but this happened : Traceback (most recent call last): File "tools/train.py", line 135, in
main()
File "tools/train.py", line 131, in main
runner.train()
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1703, in train
self._train_loop = self.build_train_loop(
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1495, in build_train_loop
loop = LOOPS.build(
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, args, kwargs, registry=self)
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(args) # type: ignore
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/runner/loops.py", line 44, in init
super().init(runner, dataloader)
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/runner/base_loop.py", line 26, in init
self.dataloader = runner.build_dataloader(
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1353, in build_dataloader
dataset = DATASETS.build(dataset_cfg)
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, args, kwargs, registry=self)
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(args) # type: ignore
File "/home/andrew/mmaction2-main/mmaction/datasets/rawframe_dataset.py", line 99, in init
super().init(
File "/home/andrew/mmaction2-main/mmaction/datasets/base.py", line 48, in init
super().init(
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 245, in init
self.full_init()
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 296, in full_init
self.data_list = self.load_data_list()
File "/home/andrew/mmaction2-main/mmaction/datasets/rawframe_dataset.py", line 121, in load_data_list
if self.data_prefix['img'] is not None:
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/config/config.py", line 136, in getitem
return self.build_lazy(super().getitem(key))
File "/home/andrew/anaconda3/envs/mmaction/lib/python3.8/site-packages/mmengine/config/config.py", line 103, in missing
raise KeyError(name)
KeyError: 'img'
dataset settings
base = [
'../../base/models/slowfast_r50.py', '../../base/default_runtime.py'
]
dataset_type = 'RawframeDataset' # lllllllllllllllllllllllllllllllllllllllll data_root = 'data/hmdb51/rawframes'
data_root_val = 'data/hmdb51/rawframes'
ann_file_train = 'data/hmdb51/hmdb51_train_split_1_rawframes.txt'
ann_file_val = 'data/hmdb51/hmdb51_val_split_1_rawframes.txt'
ann_file_test = 'data/ucf101/ucf101_train_split_1_rawframes.txt'
file_client_args = dict(io_backend='disk')
train_pipeline = [
dict(type='DecordInit', file_client_args),
dict(type='SampleFrames',
clip_len=32,
frame_interval=2,
num_clips=1),
dict(type='DecordDecode'), # llllllllllllllllllllllllll
dict(type='Resize', scale=(-1, 256)),
dict(type='RandomResizedCrop'),
dict(type='Resize', scale=(224, 224), keep_ratio=False),
dict(type='Flip', flip_ratio=0.5),
dict(type='FormatShape', input_format='NCTHW'),
dict(type='PackActionInputs')
]
val_pipeline = [
dict(type='DecordInit', file_client_args),
dict(
type='SampleFrames',
clip_len=32,
frame_interval=2,
num_clips=1,
test_mode=True),
dict(type='DecordDecode'), # llllllllllllllllllllllllll
dict(type='Resize', scale=(-1, 256)),
dict(type='CenterCrop', crop_size=224),
dict(type='FormatShape', input_format='NCTHW'),
dict(type='PackActionInputs')
]
test_pipeline = [
dict(type='DecordInit', **file_client_args),
dict(
type='SampleFrames',
clip_len=32,
frame_interval=2,
num_clips=10,
test_mode=True),
dict(type='DecordDecode'), # llllllllllllllllllllllllll
dict(type='Resize', scale=(-1, 256)),
dict(type='ThreeCrop', crop_size=256),
dict(type='FormatShape', input_format='NCTHW'),
dict(type='PackActionInputs')
]
train_dataloader = dict(
batch_size=1,
num_workers=8,
persistent_workers=True,
sampler=dict(type='DefaultSampler', shuffle=True),
dataset=dict(
type=dataset_type,
ann_file=ann_file_train,
data_prefix=dict(video=data_root),
pipeline=train_pipeline))
val_dataloader = dict(
batch_size=1,
num_workers=8,
persistent_workers=True,
sampler=dict(type='DefaultSampler', shuffle=False),
dataset=dict(
type=dataset_type,
ann_file=ann_file_val,
data_prefix=dict(video=data_root_val),
pipeline=val_pipeline,
test_mode=True))
test_dataloader = dict(
batch_size=1,
num_workers=8,
persistent_workers=True,
sampler=dict(type='DefaultSampler', shuffle=False),
dataset=dict(
type=dataset_type,
ann_file=ann_file_test,
data_prefix=dict(video=data_root_val),
pipeline=test_pipeline,
test_mode=True))
val_evaluator = dict(type='AccMetric')
test_evaluator = val_evaluator
train_cfg = dict(
type='EpochBasedTrainLoop', max_epochs=256, val_begin=1, val_interval=5)
val_cfg = dict(type='ValLoop')
test_cfg = dict(type='TestLoop')
optim_wrapper = dict(
optimizer=dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=1e-4),
clip_grad=dict(max_norm=40, norm_type=2))
param_scheduler = [
dict(
type='LinearLR',
start_factor=0.1,
by_epoch=True,
begin=0,
end=34,
convert_to_iter_based=True),
dict(
type='CosineAnnealingLR',
T_max=256,
eta_min=0,
by_epoch=True,
begin=0,
end=256)
]
default_hooks = dict(
checkpoint=dict(interval=4, max_keep_ckpts=3), logger=dict(interval=100))
model settings
model = dict( type='Recognizer3D', backbone=dict( type='ResNet3dSlowFast', pretrained=None, resample_rate=8, # tau speed_ratio=8, # alpha channel_ratio=8, # beta_inv slow_pathway=dict( type='resnet3d', depth=50, pretrained=None, lateral=True, conv1_kernel=(1, 7, 7), dilations=(1, 1, 1, 1), conv1_stride_t=1, pool1_stride_t=1, inflate=(0, 0, 1, 1), norm_eval=False), fast_pathway=dict( type='resnet3d', depth=50, pretrained=None, lateral=False, base_channels=8, conv1_kernel=(5, 7, 7), conv1_stride_t=1, pool1_stride_t=1, norm_eval=False)), cls_head=dict( type='SlowFastHead', in_channels=2304, # 2048+256 num_classes=51, # lllllllllllllllllllll spatial_type='avg', dropout_ratio=0.5, average_clips='prob'), data_preprocessor=dict( type='ActionDataPreprocessor', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], format_shape='NCTHW'))