search

mks0601 / I2L-MeshNet_RELEASE

Official PyTorch implementation of "I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image", ECCV 2020
MIT License
710 stars 130 forks source link

COCO GT fitted meshes projection problem #68

Open jackal02 opened 3 years ago

jackal02 commented 3 years ago

Hello,

I have a problem with projecting the COCO GT fitted SMPL mesh in the image coordinate space.

I want to map a different set of 2D key points, but the projected points and mesh are off. I am using the MSCOCO.py code in order to load the GT data, the smpl_mesh_cam variable alongside the corresponding cam_param.

Are there any other additional operations needed to align the 3D mesh in camera coordinate space before projecting some points in 2D? Also, is the 3D translation available in the GT data the depth information for the pelvis root joint?

In the image below, the blue points are the 2D points available in the GT (from the fitted mesh) and the red points are the ones that I am trying to project from the 3D SMPL mesh.

proof

Thank you!

mks0601 commented 3 years ago

How did you project red points? All information including root joint depth is included in the json file.

jackal02 commented 3 years ago

The red points are from the SMPL mesh in camera coordinates (smpl_mesh_cam). In order to project them, I use cam_param.

Is the translation available in the SMPL params json file the depth of the root joint ? Only the translation available in the GT is added to the mesh, I did not use the root joint depth json because it is only for the val split and I am working with the train split at the moment.

Is it necessary to also generate the root joint depth for the train split ?

mks0601 commented 3 years ago

No. Just using coco_smplifyx_train.json is enough.