search

wmcnally / kapao

KAPAO is an efficient single-stage human pose estimation model that detects keypoints and poses as objects and fuses the detections to predict human poses.
GNU General Public License v3.0
753 stars 103 forks source link

Question about keypoint fusion #78

Open BenjaminJonghyun opened 2 years ago

BenjaminJonghyun commented 2 years ago

Thanks for sharing a nice work :)

I was wondering about source codes about keypoint fusion.

In this line https://github.com/wmcnally/kapao/blob/ff66c3773eb07ceef5d0cb10a47883cf63616e5a/val.py#L116, pose keypoint coordinates are changed to keypoint object ones if keypoint object confidence > pose keypoint confidence.

From this line https://github.com/wmcnally/kapao/blob/ff66c3773eb07ceef5d0cb10a47883cf63616e5a/val.py#L101, pose keypoint confidences are set to "0". Following above codes, if the distance between pose keypoint and keypoint object < distance threshold, then most pose keypoint will be changed to keypoint object.

So, my question is that why it is necessary to compare confidences between pose keypoint and keypoint object.

Please give some comments.

wmcnally commented 2 years ago

There are many candidate keypoint objects. We keep the one with the highest confidence. If the confidence is higher than one that was already matched, we overwrite it.