open-mmlab / mmpose

OpenMMLab Pose Estimation Toolbox and Benchmark.
https://mmpose.readthedocs.io/en/latest/
Apache License 2.0
5.91k stars 1.26k forks source link

[Docs] rtmpose-s perform worse on body8 than coco! #2811

Closed robbiewongBD closed 1 year ago

robbiewongBD commented 1 year ago

📚 The doc issue

rtmpose-s achieve 72.2 on coco-val-2017 using aic+coco while achieve only 69.7 on coco-val-2017 using 7 public datasets(including aic+coco),why rtmpose-s perform worse with much data? image image

Suggest a potential alternative/fix

is it because that the other data do not have nose ear eye annotations?

Tau-J commented 1 year ago

Hi @robbiewongBD , the body8 version is trained on the naively combined dataset with few hyper-parameter tuning, which results in the non-optimal performance. But we don't have a plan to conduct any further experiments to update it, sorry for that.

robbiewongBD commented 1 year ago

@Tau-J hi, do you have any suggestions on how to tune the hyper-parameter?

Tau-J commented 1 year ago

It is normal when training lightweight models on large combined dataset and evaluate on a specific sub-dataset(e.g. COCO for Pose task). The lightweight model is too tiny to generalize and will fit more on the largest sub-dataset. Therefore, you can also change the sampling ratio of different datasets when training.