How to get MANO shape and pose parameters from meta_data?

[ZhaoLizz]

Hi! Thanks for sharing the great work! I have noticed that we can get meta_data, which contains the hand pose in terms of Bones, from the .h5 files. And there are MANO meshes in the evals folder. May I ask how to get MANO pose and shape parameters from meta_data for other frames not in eval? Thanks in advance.

[coreqode]

Hi, MANO parameters are present for grasps sequences in the mano_params directory inside each grasp sequence dir.

[ZhaoLizz]

Thank you for your timely reply.

I have found the mano_params for the grasps sequences. However, I could not locate the mano_params for the hand images in the actions_hdf5, which I intend to use to train a hand model without an object. It appears that the hand images in actions_hdf5 are segmented from the grasps with SAM and InstantNGP.

Is there a correspondence index file available to obtain the mano_params for these hand images in hdf5 from the grasps sequences? This would be extremely helpful for my research.

Thank you very much for your assistance.

[coreqode]

actions_hdf5 is not a part of grasps data. Instead, we captured it individually for each subject to fit hand model.

[ZhaoLizz]

Well, in that case, is there any way I can achieve mano_params for this individual hand-capture data? It seems you have already annotated 3D key points for these hand images but do not provide fitted mano.

[ZhaoLizz]

By the way, can you provide any hint on how to use mano_params from the grasps sequence to generate posed MANO meshes? The mano_params provide angle (33, ), which differs from the original MANO pose parameters (16, 3).

[coreqode]

We optimize for PCA components directly. So there are PCA components. You can still get the MANO mesh using this. I will try to update the direction to use MANO and other helper files to use dataset by this weekend.

[coreqode]

Sorry for delay in response. Please check scripts/dataset_helpers/load_videos.py file on using MANO parameters.