search

hongsukchoi / Pose2Mesh_RELEASE

Official Pytorch implementation of "Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose", ECCV 2020
MIT License
678 stars 69 forks source link

How to deal with multiple people in a single image with some errors of estimated poses? #26

Open Qsminefor opened 3 years ago

Qsminefor commented 3 years ago

Thank you for your sharing your great work, and hope your another work '3d crowdnet' will be published on the top conference. I am curious about how to deal with the errors of pose estimations, especially for some bottom-top methods, such as higher hrnet, in multi-person condition.

hongsukchoi commented 3 years ago

Hi @Qsminefor,

Thank you for your interest in my works! Pose2Mesh can already deal with some errors in a 2D pose, since it is trained with noisy 2D poses, which mimic the errors of 2D pose estimators like HigherHRNet. Please refer to Secion3.1 in the paper for more details.

However, currently Pose2mesh cannot successfully deal with missing joints of 2D pose estimators. For example, if some 2D joint predictions have coordinates like (0,0), it would lead to a big error. If a user wants a plausible output, one should fill the missing joints with plausible 2D coordinates first.

Qsminefor commented 3 years ago

Thanks for your quick response. Sorry for the misunderstanding. My concern is how to deal with the error number of multiple people. For example, Higher hrnet may estimate seven people, even there are only five people.

hongsukchoi commented 3 years ago

I see. I used some heuristics per an image. For example, you can exclude 2D pose outputs with many missing joints (joints with very low confidence).

NMS could be also an option to prevent duplicates.

Qsminefor commented 3 years ago

Yes，I see. But I still think these operations cannot prove the number of people is correct. I will try it. And could you give me more details about this?