search

facebookresearch / VideoPose3D

Efficient 3D human pose estimation in video using 2D keypoint trajectories
Other
3.75k stars 757 forks source link

Predictions on occluded detections #126

Open AlbertoSabater opened 4 years ago

AlbertoSabater commented 4 years ago

I am performing 3D pose inference from a video where half of the body of a person is occluded by a table. First, I extract 2D keypoints with Detectron, that gave me low scoring confidences for occluded keypoints. Occluded keypoints present undesired locations. To get the 3D inference I feed VideoPose3D with all inferred 2D points which lead with wrong predictions for the occluded keypoints.

My question is if the 2D outlier keypoints affect the rest of keypoints. Is it possible to get predictions only for not-occluded keypoints without retraining? Occluded keypoints can be hardcoded before hand.

Thanks in advance!

dariopavllo commented 4 years ago

The position of a keypoint is influenced by the other keypoints, and the pretrained model was trained without occlusions. If you want the best results, you should retrain the model with the new set of keypoints.

yt2639 commented 2 years ago

I am performing 3D pose inference from a video where half of the body of a person is occluded by a table. First, I extract 2D keypoints with Detectron, that gave me low scoring confidences for occluded keypoints. Occluded keypoints present undesired locations. To get the 3D inference I feed VideoPose3D with all inferred 2D points which lead with wrong predictions for the occluded keypoints.

My question is if the 2D outlier keypoints affect the rest of keypoints. Is it possible to get predictions only for not-occluded keypoints without retraining? Occluded keypoints can be hardcoded before hand.

Thanks in advance!

Hi @AlbertoSabater , thanks for your question! I am facing exactly the same question that you had. How did you end up solving the issue? or did you simply change to use another keypoint predictor? Thanks!