search

open-mmlab / mmpose

OpenMMLab Pose Estimation Toolbox and Benchmark.
https://mmpose.readthedocs.io/en/latest/
Apache License 2.0
5.59k stars 1.22k forks source link

[Bug] KeyError: 'inputs' #2499

Closed eqwwadfw closed 1 year ago

eqwwadfw commented 1 year ago

Prerequisite

Environment

OrderedDict([('sys.platform', 'linux'), ('Python', '3.10.10 (main, Mar 21 2023, 18:45:11) [GCC 11.2.0]'), ('CUDA available', False), ('numpy_random_seed', 2147483648), ('GCC', 'gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)'), ('PyTorch', '2.0.1+cu117'), ('PyTorch compiling details', 'PyTorch built with:\n - GCC 9.3\n - C++ Version: 201703\n - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications\n - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n - OpenMP 201511 (a.k.a. OpenMP 4.5)\n - LAPACK is enabled (usually provided by MKL)\n - NNPACK is enabled\n - CPU capability usage: AVX2\n - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n'), ('TorchVision', '0.15.2+cu117'), ('OpenCV', '4.7.0'), ('MMEngine', '0.7.4'), ('MMPose', '1.0.0+59eff49')])

mmcv 2.0.0 mmdet 3.0.0 /home/jiao_yang/mmdetection mmengine 0.7.4 mmpose 1.0.0 /home/jiao_yang/mmpose mmpycocotools 12.0.3

Reproduces the problem - code sample

coco.py： dataset_info = dict( dataset_name='coco', paper_info=dict( author='Lin, Tsung-Yi and Maire, Michael and ' 'Belongie, Serge and Hays, James and ' 'Perona, Pietro and Ramanan, Deva and ' r'Doll{\'a}r, Piotr and Zitnick, C Lawrence', title='Microsoft coco: Common objects in context', container='European conference on computer vision', year='2014', homepage='http://cocodataset.org/', ), keypoint_info={
0: dict(name='head', id=0, color=[166, 43, 30], type='upper', swap=''), 1: dict(name='front', id=1, color=[212, 85, 26], type='upper', swap=''), 2: dict(name='middle', id=2, color=[245, 183, 15], type='upper', swap=''), 3: dict(name='back', id=3, color=[32, 124, 77], type='upper', swap=''), 4: dict(name='tail', id=4, color=[7, 82, 124], type='upper', swap='') }, skeleton_info={ 0: dict(link=('head', 'front'), id=0, color=[129, 58, 107]), 1: dict(link=('front', 'middle'), id=1, color=[129, 58, 107]), 2: dict(link=('middle', 'back'), id=2, color=[129, 58, 107]), 3: dict(link=('back', 'tail'), id=3, color=[129, 58, 107]) }, joint_weights=[ 1., 1., 1., 1., 1. ], sigmas=[ 0.025, 0.035, 0.025, 0.055, 0.035 ]) coco_dataset.py:

Copyright (c) OpenMMLab. All rights reserved.

from mmpose.registry import DATASETS from ..base import BaseCocoStyleDataset

@DATASETS.register_module() class CocoDataset(BaseCocoStyleDataset): """COCO dataset for pose estimation.

"Microsoft COCO: Common Objects in Context", ECCV'2014.
More details can be found in the `paper
<https://arxiv.org/abs/1405.0312>`__ .

COCO keypoints::

    0: 'nose',
    1: 'left_eye',
    2: 'right_eye',
    3: 'left_ear',
    4: 'right_ear',
    5: 'left_shoulder',
    6: 'right_shoulder',
    7: 'left_elbow',
    8: 'right_elbow',
    9: 'left_wrist',
    10: 'right_wrist',
    11: 'left_hip',
    12: 'right_hip',
    13: 'left_knee',
    14: 'right_knee',
    15: 'left_ankle',
    16: 'right_ankle'

Args:
    ann_file (str): Annotation file path. Default: ''.
    bbox_file (str, optional): Detection result file path. If
        ``bbox_file`` is set, detected bboxes loaded from this file will
        be used instead of ground-truth bboxes. This setting is only for
        evaluation, i.e., ignored when ``test_mode`` is ``False``.
        Default: ``None``.
    data_mode (str): Specifies the mode of data samples: ``'topdown'`` or
        ``'bottomup'``. In ``'topdown'`` mode, each data sample contains
        one instance; while in ``'bottomup'`` mode, each data sample
        contains all instances in a image. Default: ``'topdown'``
    metainfo (dict, optional): Meta information for dataset, such as class
        information. Default: ``None``.
    data_root (str, optional): The root directory for ``data_prefix`` and
        ``ann_file``. Default: ``None``.
    data_prefix (dict, optional): Prefix for training data. Default:
        ``dict(img=None, ann=None)``.
    filter_cfg (dict, optional): Config for filter data. Default: `None`.
    indices (int or Sequence[int], optional): Support using first few
        data in annotation file to facilitate training/testing on a smaller
        dataset. Default: ``None`` which means using all ``data_infos``.
    serialize_data (bool, optional): Whether to hold memory using
        serialized objects, when enabled, data loader workers can use
        shared RAM from master process instead of making a copy.
        Default: ``True``.
    pipeline (list, optional): Processing pipeline. Default: [].
    test_mode (bool, optional): ``test_mode=True`` means in test phase.
        Default: ``False``.
    lazy_init (bool, optional): Whether to load annotation during
        instantiation. In some cases, such as visualization, only the meta
        information of the dataset is needed, which is not necessary to
        load annotation file. ``Basedataset`` can skip load annotations to
        save time by set ``lazy_init=False``. Default: ``False``.
    max_refetch (int, optional): If ``Basedataset.prepare_data`` get a
        None img. The maximum extra number of cycles to get a valid
        image. Default: 1000.
"""

METAINFO: dict = dict(from_file='configs/_base_/datasets/coco.py')

worm_config.py: base = [ '../../../base/default_runtime.py', '../../../base/datasets/coco.py' ]

runtime

train_cfg = dict(max_epochs=300, val_interval=10)

optimizer

optim_wrapper = dict(optimizer=dict( type='Adam', lr=1.5e-3, ))

learning policy

param_scheduler = [ dict( type='LinearLR', begin=0, end=500, start_factor=0.001, by_epoch=False), # warm-up dict( type='MultiStepLR', begin=0, end=300, milestones=[200, 260], gamma=0.1, by_epoch=True) ]

automatically scaling LR based on the actual training batch size

auto_scale_lr = dict(base_batch_size=192)

hooks

default_hooks = dict( checkpoint=dict(save_best='coco/AP', rule='greater', interval=50))

codec settings

codec = dict( type='AssociativeEmbedding', input_size=(1024, 1024), heatmap_size=(128, 128), sigma=2, decode_keypoint_order=[ 0, 1, 2, 3, 4 ], decode_max_instances=30)

model settings

model = dict( type='BottomupPoseEstimator', data_preprocessor=dict( type='PoseDataPreprocessor', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], bgr_to_rgb=True), backbone=dict( type='HRNet', in_channels=3, extra=dict( stage1=dict( num_modules=1, num_branches=1, block='BOTTLENECK', num_blocks=(4, ), num_channels=(64, )), stage2=dict( num_modules=1, num_branches=2, block='BASIC', num_blocks=(4, 4), num_channels=(32, 64)), stage3=dict( num_modules=4, num_branches=3, block='BASIC', num_blocks=(4, 4, 4), num_channels=(32, 64, 128)), stage4=dict( num_modules=3, num_branches=4, block='BASIC', num_blocks=(4, 4, 4, 4), num_channels=(32, 64, 128, 256))), init_cfg=dict( type='Pretrained', checkpoint='https://download.openmmlab.com/mmpose/' 'pretrain_models/hrnet_w32-36af842e.pth'), ), head=dict( type='AssociativeEmbeddingHead', in_channels=32, num_keypoints=5, tag_dim=1, tag_per_keypoint=True, deconv_out_channels=None, keypoint_loss=dict(type='KeypointMSELoss', use_target_weight=True), tag_loss=dict(type='AssociativeEmbeddingLoss', loss_weight=0.001),

The heatmap will be resized to the input size before decoding

    # if ``restore_heatmap_size==True``
    decoder=dict(codec, heatmap_size=codec['input_size'])),
test_cfg=dict(
    multiscale_test=False,
    flip_test=True,
    shift_heatmap=True,
    restore_heatmap_size=True,
    align_corners=False))

base dataset settings

dataset_type = 'CocoDataset' data_mode = 'bottomup' data_root = 'data/worm/'

pipelines

train_pipeline = [] val_pipeline = [ dict(type='LoadImage'), dict( type='BottomupResize', input_size=codec['input_size'], size_factor=32, resize_mode='expand'), dict(type='PackPoseInputs') ]

data loaders

train_dataloader = dict( batch_size=1, num_workers=2, persistent_workers=True, sampler=dict(type='DefaultSampler', shuffle=True), dataset=dict( type=dataset_type, data_root=data_root, data_mode=data_mode, ann_file='annotations-new/worm_train.json', data_prefix=dict(img='data/'), pipeline=train_pipeline, )) val_dataloader = dict( batch_size=1, num_workers=2, persistent_workers=True, drop_last=False, sampler=dict(type='DefaultSampler', shuffle=False, round_up=False), dataset=dict( type=dataset_type, data_root=data_root, data_mode=data_mode, ann_file='annotations-new/worm_val.json', data_prefix=dict(img='data/'), test_mode=True, pipeline=val_pipeline, )) test_dataloader = val_dataloader

evaluators

val_evaluator = dict( type='CocoMetric', ann_file=data_root + 'annotations-new/worm_val.json', nms_mode='none', score_mode='keypoint', ) test_evaluator = val_evaluator

Reproduces the problem - command or script

python tools/train.py configs/body_2d_keypoint/associative_embedding/coco/worm_config2.py

Reproduces the problem - error message

Traceback (most recent call last): File "/home/jiao_yang/mmpose/tools/train.py", line 160, in main() File "/home/jiao_yang/mmpose/tools/train.py", line 156, in main runner.train() File "/home/jiao_yang/miniconda3/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1721, in train model = self.train_loop.run() # type: ignore File "/home/jiao_yang/miniconda3/lib/python3.10/site-packages/mmengine/runner/loops.py", line 96, in run self.run_epoch() File "/home/jiao_yang/miniconda3/lib/python3.10/site-packages/mmengine/runner/loops.py", line 112, in run_epoch self.run_iter(idx, data_batch) File "/home/jiao_yang/miniconda3/lib/python3.10/site-packages/mmengine/runner/loops.py", line 128, in run_iter outputs = self.runner.model.train_step( File "/home/jiao_yang/miniconda3/lib/python3.10/site-packages/mmengine/model/base_model/base_model.py", line 113, in train_step data = self.data_preprocessor(data, True) File "/home/jiao_yang/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/jiao_yang/miniconda3/lib/python3.10/site-packages/mmengine/model/base_model/data_preprocessor.py", line 247, in forward _batch_inputs = data['inputs'] KeyError: 'inputs'

Additional information

I want to run my data set, and put the data set in the path of data/worm/, and configure worm_config.py according to the official document, and put it in the path of configs/body_2d_keypoint/associative_embedding/coco/, In addition, the coco.py file under the configs/base/datasets/ path was modified to adapt it to my dataset, but when running the command python tools/train.py configs/body_2d_keypoint/associative_embedding/coco/worm_config2.py When KeyError: 'inputs' is displayed, why is it? I have already configured the path of the file and the path of the data set.

Tau-J commented 1 year ago

Hi @eqwwadfw , Associative Embedding is still under migration, you can refer to here to consult the progress, or you should switch to 0.x branch to use the old version.

eqwwadfw commented 1 year ago

Thank you for your answer, how many of the algorithms in configs/body_2d_keypoint have been migrated, because I want to study the trajectory of the body on 2D, so I want to use these algorithms in version 1.0.

eqwwadfw commented 1 year ago

你好@eqwwadfw，Associative Embedding 还在迁移中，可以参考这里咨询进度，或者应该切换到0.x分支使用旧版本。

Thank you for your answer, how many of the algorithms in configs/body_2d_keypoint have been migrated, because I want to study the trajectory of the body on 2D, so I want to use these algorithms in version 1.0.

Tau-J commented 1 year ago

Please refer to the link above

eqwwadfw commented 1 year ago

Thank you very much!