Tau-J
commented
1 year ago @vicnoah , 你好，谢谢使用 MMPose。当自定义数据集的关键点格式不等于17时，请修改 config 中 model 下 head 部分的 out_channels=17 项，改成你的数据集点数
Closed vicnoah closed 1 year ago
Tau-J
commented
1 year ago @vicnoah , 你好，谢谢使用 MMPose。当自定义数据集的关键点格式不等于17时，请修改 config 中 model 下 head 部分的 out_channels=17 项，改成你的数据集点数
vicnoah
commented
1 year ago @vicnoah , 你好，谢谢使用 MMPose。当自定义数据集的关键点格式不等于17时，请修改 config 中 model 下 head 部分的
out_channels=17项，改成你的数据集点数
好的，谢谢
vicnoah
commented
1 year ago `> @vicnoah , 你好，谢谢使用 MMPose。当自定义数据集的关键点格式不等于17时，请修改 config 中 model 下 head 部分的out_channels=17` 项，改成你的数据集点数
你好，按照你给的方法，前面的报错确实解决了。不过执行到模型保存时会报另一个错误，我在issue里面看到有一个同样的问题。他自己解决了，说是数据集存在问题，但没有具体的信息。我仔细对照了文档发现我少了一个area标注，其它标注都没有发现明显的问题。
这是我的报错信息，请问这是什么问题呢？
09/21 17:56:55 - mmengine - INFO - Exp name: rtmpose-m_8xb256-420e_coco-256x192_rail_20230921_153504
09/21 17:56:55 - mmengine - INFO - Saving checkpoint at 10 epochs
/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py:560: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 8, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
09/21 17:57:20 - mmengine - INFO - Epoch(val) [10][ 50/323] eta: 0:02:09 time: 0.473742 data_time: 0.187716 memory: 4561
09/21 17:57:41 - mmengine - INFO - Epoch(val) [10][100/323] eta: 0:01:39 time: 0.418373 data_time: 0.145407 memory: 822
09/21 17:58:01 - mmengine - INFO - Epoch(val) [10][150/323] eta: 0:01:14 time: 0.398568 data_time: 0.114515 memory: 822
09/21 17:58:21 - mmengine - INFO - Epoch(val) [10][200/323] eta: 0:00:52 time: 0.403443 data_time: 0.120536 memory: 822
09/21 17:58:40 - mmengine - INFO - Epoch(val) [10][250/323] eta: 0:00:30 time: 0.366690 data_time: 0.076499 memory: 822
09/21 17:58:59 - mmengine - INFO - Epoch(val) [10][300/323] eta: 0:00:09 time: 0.393036 data_time: 0.108918 memory: 822
09/21 17:59:12 - mmengine - INFO - Evaluating CocoMetric...
Loading and preparing results...
DONE (t=0.57s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *keypoints*
Traceback (most recent call last):
File "/content/mmpose/tools/train.py", line 161, in <module>
main()
File "/content/mmpose/tools/train.py", line 157, in main
runner.train()
File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/runner.py", line 1745, in train
model = self.train_loop.run() # type: ignore
File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/loops.py", line 102, in run
self.runner.val_loop.run()
File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/loops.py", line 366, in run
metrics = self.evaluator.evaluate(len(self.dataloader.dataset))
File "/usr/local/lib/python3.10/dist-packages/mmengine/evaluator/evaluator.py", line 79, in evaluate
_results = metric.evaluate(size)
File "/usr/local/lib/python3.10/dist-packages/mmengine/evaluator/metric.py", line 133, in evaluate
_metrics = self.compute_metrics(results) # type: ignore
File "/content/mmpose/mmpose/evaluation/metrics/coco_metric.py", line 440, in compute_metrics
info_str = self._do_python_keypoint_eval(outfile_prefix)
File "/content/mmpose/mmpose/evaluation/metrics/coco_metric.py", line 502, in _do_python_keypoint_eval
coco_eval.evaluate()
File "/usr/local/lib/python3.10/dist-packages/xtcocotools/cocoeval.py", line 257, in evaluate
self.ious = {(imgId, catId): computeIoU(imgId, catId) \
File "/usr/local/lib/python3.10/dist-packages/xtcocotools/cocoeval.py", line 257, in <dictcomp>
self.ious = {(imgId, catId): computeIoU(imgId, catId) \
File "/usr/local/lib/python3.10/dist-packages/xtcocotools/cocoeval.py", line 373, in computeOks
dx = xd - xg
ValueError: operands could not be broadcast together with shapes (4,) (2,3) 可以检查一下你定义的metainfo里sigmas是否数量跟关键点数一致
vicnoah
commented
1 year ago 可以检查一下你定义的metainfo里sigmas是否数量跟关键点数一致
四个关键点，sigmas是四个。我为了先测试训练过程就随便填了几个值进去，没有去计算sigmas。不知道是不是这个原因。
这是我这一部分的配置信息 joint_weights=[ 1., 1., 1., 1. ], sigmas=[ 0.025, 0.025, 0.025, 0.025 ])
vicnoah
commented
1 year ago 可以检查一下你定义的metainfo里sigmas是否数量跟关键点数一致
这是我的完整配置文件，我是在已有的配置基础上改的。 我不太明白是什么原因造成的问题。希望项目能够有一个更加完整的自定义数据集及关键点的文档。 我看到rtmpose-m原始配置中也有一个sigma部分，不过原来的配置就只有两个值。
数据集格式:
{
"images": [
{
"file_name": "1.jpg",
"id": 1,
"width": 1920,
"height": 1200
}
],
"categories": [
{"id": 1, "name": "box"}
],
"annotations": [
{
"id": 1,
"image_id": 1,
"category_id": 1,
"bbox": [
733,
368,
50,
51
],
"iscrowd": 0,
"area": 2474.5,
"num_keypoints": 4,
"segmentation": [
[
734,
369,
783,
368,
783,
418,
733,
419
]
],
"keypoints": [
[
735,
370,
2
],
[
782,
369,
2
],
[
782,
417,
2
],
[
734,
418,
2
]
]
}
]
}base.py
dataset_info = dict(
dataset_name='coco',
paper_info=[
dict(
author='Lin, Tsung-Yi and Maire, Michael and '
'Belongie, Serge and Hays, James and '
'Perona, Pietro and Ramanan, Deva and '
r'Doll{\'a}r, Piotr and Zitnick, C Lawrence',
title='Microsoft coco: Common objects in context',
container='European conference on computer vision',
year='2014',
homepage='http://cocodataset.org/',
),
dict(
author='Wu, Jiahong and Zheng, He and Zhao, Bo and '
'Li, Yixin and Yan, Baoming and Liang, Rui and '
'Wang, Wenjia and Zhou, Shipei and Lin, Guosen and '
'Fu, Yanwei and others',
title='Ai challenger: A large-scale dataset for going '
'deeper in image understanding',
container='arXiv',
year='2017',
homepage='https://github.com/AIChallenger/AI_Challenger_2017',
),
],
keypoint_info={
0:
dict(name='left_top', id=0, color=[51, 153, 255], type='upper', swap='right_top'),
1:
dict(
name='right_top',
id=1,
color=[51, 153, 255],
type='upper',
swap='left_top'),
2:
dict(
name='right_down',
id=2,
color=[51, 153, 255],
type='lower',
swap='left_down'),
3:
dict(
name='left_down',
id=3,
color=[51, 153, 255],
type='lower',
swap='right_down'),
},
skeleton_info={
0:
dict(link=('left_top', 'right_top'), id=0, color=[0, 255, 0]),
1:
dict(link=('right_top', 'right_down'), id=1, color=[0, 255, 0]),
2:
dict(link=('right_down', 'left_down'), id=2, color=[255, 128, 0]),
3:
dict(link=('left_down', 'left_top'), id=3, color=[255, 128, 0]),
},
joint_weights=[
1., 1., 1., 1.
],
sigmas=[
0.025, 0.025, 0.025, 0.025
])
rtmpose-m.py
_base_ = ['../../../_base_/default_runtime.py']
# runtime
max_epochs = 10
stage2_num_epochs = 2
base_lr = 4e-3
train_cfg = dict(max_epochs=max_epochs, val_interval=10)
randomness = dict(seed=21)
# optimizer
optim_wrapper = dict(
type='OptimWrapper',
optimizer=dict(type='AdamW', lr=base_lr, weight_decay=0.05),
paramwise_cfg=dict(
norm_decay_mult=0, bias_decay_mult=0, bypass_duplicate=True))
# learning rate
param_scheduler = [
dict(
type='LinearLR',
start_factor=1.0e-5,
by_epoch=False,
begin=0,
end=1000),
dict(
# use cosine lr from 210 to 420 epoch
type='CosineAnnealingLR',
eta_min=base_lr * 0.05,
begin=max_epochs // 2,
end=max_epochs,
T_max=max_epochs // 2,
by_epoch=True,
convert_to_iter_based=True),
]
# automatically scaling LR based on the actual training batch size
auto_scale_lr = dict(base_batch_size=1024)
# codec settings
codec = dict(
type='SimCCLabel',
input_size=(192, 256),
sigma=(4.9, 5.66),
simcc_split_ratio=2.0,
normalize=False,
use_dark=False)
# model settings
model = dict(
type='TopdownPoseEstimator',
data_preprocessor=dict(
type='PoseDataPreprocessor',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
bgr_to_rgb=True),
backbone=dict(
_scope_='mmdet',
type='CSPNeXt',
arch='P5',
expand_ratio=0.5,
deepen_factor=0.67,
widen_factor=0.75,
out_indices=(4, ),
channel_attention=True,
norm_cfg=dict(type='SyncBN'),
act_cfg=dict(type='SiLU'),
init_cfg=dict(
type='Pretrained',
prefix='backbone.',
checkpoint='https://download.openmmlab.com/mmpose/v1/projects/'
'rtmposev1/cspnext-m_udp-aic-coco_210e-256x192-f2f7d6f6_20230130.pth' # noqa
)),
head=dict(
type='RTMCCHead',
in_channels=768,
out_channels=4,
input_size=codec['input_size'],
in_featuremap_size=tuple([s // 32 for s in codec['input_size']]),
simcc_split_ratio=codec['simcc_split_ratio'],
final_layer_kernel_size=7,
gau_cfg=dict(
hidden_dims=256,
s=128,
expansion_factor=2,
dropout_rate=0.,
drop_path=0.,
act_fn='SiLU',
use_rel_bias=False,
pos_enc=False),
loss=dict(
type='KLDiscretLoss',
use_target_weight=True,
beta=10.,
label_softmax=True),
decoder=codec),
test_cfg=dict(flip_test=True))
# base dataset settings
dataset_type = 'CocoDataset'
data_mode = 'topdown'
data_root = 'data/coco/'
backend_args = dict(backend='local')
# backend_args = dict(
# backend='petrel',
# path_mapping=dict({
# f'{data_root}': 's3://openmmlab/datasets/detection/coco/',
# f'{data_root}': 's3://openmmlab/datasets/detection/coco/'
# }))
# pipelines
train_pipeline = [
dict(type='LoadImage', backend_args=backend_args),
dict(type='GetBBoxCenterScale'),
dict(type='RandomFlip', direction='horizontal'),
dict(type='RandomHalfBody'),
dict(
type='RandomBBoxTransform', scale_factor=[0.6, 1.4], rotate_factor=80),
dict(type='TopdownAffine', input_size=codec['input_size']),
dict(type='mmdet.YOLOXHSVRandomAug'),
dict(
type='Albumentation',
transforms=[
dict(type='Blur', p=0.1),
dict(type='MedianBlur', p=0.1),
dict(
type='CoarseDropout',
max_holes=1,
max_height=0.4,
max_width=0.4,
min_holes=1,
min_height=0.2,
min_width=0.2,
p=1.),
]),
dict(type='GenerateTarget', encoder=codec),
dict(type='PackPoseInputs')
]
val_pipeline = [
dict(type='LoadImage', backend_args=backend_args),
dict(type='GetBBoxCenterScale'),
dict(type='TopdownAffine', input_size=codec['input_size']),
dict(type='PackPoseInputs')
]
train_pipeline_stage2 = [
dict(type='LoadImage', backend_args=backend_args),
dict(type='GetBBoxCenterScale'),
dict(type='RandomFlip', direction='horizontal'),
dict(type='RandomHalfBody'),
dict(
type='RandomBBoxTransform',
shift_factor=0.,
scale_factor=[0.75, 1.25],
rotate_factor=60),
dict(type='TopdownAffine', input_size=codec['input_size']),
dict(type='mmdet.YOLOXHSVRandomAug'),
dict(
type='Albumentation',
transforms=[
dict(type='Blur', p=0.1),
dict(type='MedianBlur', p=0.1),
dict(
type='CoarseDropout',
max_holes=1,
max_height=0.4,
max_width=0.4,
min_holes=1,
min_height=0.2,
min_width=0.2,
p=0.5),
]),
dict(type='GenerateTarget', encoder=codec),
dict(type='PackPoseInputs')
]
# data loaders
train_dataloader = dict(
batch_size=64,
num_workers=10,
persistent_workers=True,
sampler=dict(type='DefaultSampler', shuffle=True),
dataset=dict(
type=dataset_type,
data_root=data_root,
data_mode=data_mode,
ann_file='annotations/trainval.json',
data_prefix=dict(img='images/'),
metainfo=dict(from_file='configs/_base_/datasets/rail.py'),
pipeline=train_pipeline,
))
val_dataloader = dict(
batch_size=64,
num_workers=10,
persistent_workers=True,
drop_last=False,
sampler=dict(type='DefaultSampler', shuffle=False, round_up=False),
dataset=dict(
type=dataset_type,
data_root=data_root,
data_mode=data_mode,
ann_file='annotations/test.json',
# bbox_file=f'{data_root}person_detection_results/'
# 'COCO_val2017_detections_AP_H_56_person.json',
data_prefix=dict(img='images/'),
metainfo=dict(from_file='configs/_base_/datasets/rail.py'),
test_mode=True,
pipeline=val_pipeline,
))
test_dataloader = val_dataloader
# hooks
default_hooks = dict(
checkpoint=dict(save_best='coco/AP', interval=2, rule='greater', max_keep_ckpts=1))
custom_hooks = [
dict(
type='EMAHook',
ema_type='ExpMomentumEMA',
momentum=0.0002,
update_buffers=True,
priority=49),
dict(
type='mmdet.PipelineSwitchHook',
switch_epoch=max_epochs - stage2_num_epochs,
switch_pipeline=train_pipeline_stage2)
]
# evaluators
val_evaluator = dict(
type='CocoMetric',
ann_file=data_root + 'annotations/test.json')
test_evaluator = val_evaluator可以检查一下你定义的metainfo里sigmas是否数量跟关键点数一致
解决了是我的标注文件keypoints文件多了一层的问题，一直没有注意到。非常感谢
zhenhao-huang
commented
3 months ago 可以检查一下你定义的metainfo里sigmas是否数量跟关键点数一致
如何确定sigmas的大小@Tau-J
zhenhao-huang
commented
3 months ago joint_weights数量需要和关键点数一致吗@Tau-J
vicnoah
commented
3 months ago joint_weights数量需要和关键点数一致吗@Tau-J
需要一致
zhenhao-huang
commented
3 months ago joint_weights数量需要和关键点数一致吗@Tau-J
需要一致
sigmas和joint_weights大小如何确定 @Tau-J @vicnoah
vicnoah
commented
3 months ago joint_weights数量需要和关键点数一致吗@Tau-J
需要一致
sigmas和joint_weights大小如何确定 @Tau-J @vicnoah
我不懂，有文章介绍。我训练时全用的默认值，训练效果也很好
📚 The doc issue
Suggest a potential alternative/fix
No response