search

Vegetebird / StridedTransformer-Pose3D

[TMM 2022] Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation
MIT License
335 stars 37 forks source link

Back projection #18

Closed iulsko closed 1 year ago

iulsko commented 2 years ago

is there a way to back-project estimated 3D points back onto the image?

Vegetebird commented 2 years ago

You can refer to "https://github.com/facebookresearch/VideoPose3D/blob/1afb1ca0f1237776518469876342fc8669d3f6a9/run.py#L373"

iulsko commented 2 years ago

@Vegetebird hi Wenhao, yes, I have been referring to this bit but since output_3D in here is given in relative 3D coordinates (relative to the root) and not in camera 3D coordinates, the back projections do not work.

is there any way i can bring output_3D to camera 3D frame?

Vegetebird commented 2 years ago

You can use the root coordinates of gt_3d to get the absolute coordinates.

iulsko commented 2 years ago

@Vegetebird so I assume it is not possible for "in-the-wild" videos, since I do not have gt_3D?

Vegetebird commented 2 years ago

Yeah, and we do not have the camera parameter to perform projection

iulsko commented 2 years ago

@Vegetebird again, just to confirm: so we only have 3D movements in the "relative" coordinate frame. e.g. without the camera parameters we are also not able to find the absolute movement in mm/cm/m?

Vegetebird commented 2 years ago

Our method can not find the absolute movement, but you can refer to VideoPose3D to train a trajectory model to get the global positions.