Location discrepancy in 6fps.npz

[davidpagnon]

Dear Bedlam team and @pixelite1201 .

I am working with a subset of BEDLAM, namely 20221010_3-10_500_batch01hand_zoom_suburb_d_6fps.npz.

I am trying to import meshes and corresponding cameras in Blender. The rotations look okay, but the translations seem to be off, both for the cameras and the meshes. If I manually translate either the mesh or the camera, I can get a correct overlay (see screenshot), but I'm not sure how to find this translation from the npz data.

By the way, out of curiosity, is there a reason why for two persons with the same imgname, the camera translation is not set to be the same? I feel like it would have been easier to just translate the mesh. How do you compose the translations from the mesh and the camera? Example:

• npz_data['imgname'][62] -> 'seq_000000/seq_000000_0125.png' npz_data['cam_ext'][62]][...,3] -> array([0.38374163, 0.40568286, 8.35000985, 1. ])
• npz_data['imgname'][88] -> 'seq_000000/seq_000000_0125.png' npz_data['cam_ext'][88][...,3] ->array([-0.02659588, -0.07426938, 11.27912429, 1. ])

Thank you in advance!

[pixelite1201]

Hello,

6fps.npz has not been created for loading the image and projection in Blender. It has been created for training the HPS model using crops of humans. Since, during training we need to know the translation of each person with respect to camera, hence the translation value is different for each subject. If you need to understand how exactly the projection works, please check out this code

If you need to work with Blender, I would suggest you use the original 30fps.npz and load every 5th frame to get 6fps output.

[davidpagnon]

Oh okay I get it, thanks! I originally wanted to work with the 6fps dataset for storage considerations, but I'll make space and get a subset of the 30 fps one.

[davidpagnon]

Sorry for reopening it, I'm still kind of fuzzy about why and how the camera extrinsic parameters change from person to person on the same image. Also, I would have expected npz_params['pose_world'][:3] (coordinates of the root joint of the body) to be the same value as npz_params['trans_world'] (coordinates of the body).

In any case, since this is what I ultimately need, is there a way to have the camera calibration in the world coordinate system, and idem for each SMPL-X mesh?