Arthur151 / ROMP

Monocular, One-stage, Regression of Multiple 3D People and their 3D positions & trajectories in camera & global coordinates. ROMP[ICCV21], BEV[CVPR22], TRACE[CVPR2023]
https://www.yusun.work/
Apache License 2.0
1.33k stars 229 forks source link

Details about existence in 3d pose estimation #48

Closed NoLookDefense closed 3 years ago

NoLookDefense commented 3 years ago

Hello. I am coming from another details about the implementation. In 2d pose estimation, we always use heatmap regression method, which generates a third attribute "confidence" which can be used for detected non-existent keypoints. As for the keypoint whose maximum value of the whole heatmap is too low, the corresponding key-point can be recognised as "not exist", which is maybe out of the image or occluded. However, in 3d pose estimation, there's neither an attribute called "confidence", nor a heatmap. Does it means that every point are assumed to be existent? Are there any tricks for detecting the occluded body-parts?

Arthur151 commented 3 years ago

Yes, I think there are some differences between 2D and 3D pose estimation. From my view, 2D pose estimation is supposed to detect the keypoints in the input 2D images. While 3D pose estimation is supposed to retrieve the real motion state of each person appeared in the images. As for the motion state of a person, we tend to estimate the 3D location/rotation of all his/her joints. This is just my insight. Existing methods usually estimate the motion state of the whole body based on the visible body parts. Estimating the occluded body parts is based on a priori knowledge learnt from the training data. Glad to discuss these interesting things. Best~