Controlling mano hand model

[ksh960910]

Hi Christsa, While trying to make changes in mano hand model, I'm currently struggling with which part of the code should I change in order to make hand look more like human hand in the simulation. In simulation, it looks like joints are apart and the skinning of the joint part is a bit lacking. It would be grateful if you let me know where to change in the code.

Best regards, Joe

[christsa]

Hi Joe,

In rigid body simulations, there is no way to articulate a watertight mesh directly. Hence, we need to separate each joint and create a corresponding mesh to transfer it to simulation. To create a mesh, we use the skinning weights of the MANO model to assign vertices to each hand joint, which naturally causes some gaps because the vertices in-between joints are not connected (also see our appendix for more details). The hand model is defined here, where we assign the meshes to their respective joints.

In our paper, we transferred the simulated hand back to the original MANO model in post-processing to visualize a watertight mesh.

Let me know if you have any more questions.

Best, Sammy

[ksh960910]

Hi Sammy,

Thanks for your quick reply. Since I would like to try visualizing a watertight mesh, I would appreciate if you let me know some details of "transferring the simulated hand back to the original MANO model" part.

Best regards, Joe

[christsa]

Hi Joe,

Sure. What you need to do is take the hand pose from simulation and feed them through MANO here. Here's a step by step description:

1. Extract global wrist 6DoF wrist pose and local finger joint angles (51 DoF) from simulation.
2. MANO uses pca representation per default, so you need to set it to False and convert the Euler angles from the simulation to axis-angle representation. This you can easily do with scipy: from scipy.spatial.transform import Rotation as R R.from_euler('XYZ', qpos[i,6:].reshape(15, 3)).as_rotvec().reshape(-1,45)
3. Also note that the order of the pinky and ring finger are in reversed order in MANO, so you need to change these before passing it to MANO.
4. The output of MANO will give you the vertices (see their minimal example), which you can then use to render the watertight mesh using any standard visualization package. In our case, we used Blender.

Let me know if anything is unclear.

Best, Sammy

[ksh960910]

Hi Sammy,

Thanks for the detailed explanation. I really appreciate it.

Best regards, Joe

[christsa]

Happy to help.

Feel free to reopen the issue if you get stuck.

Best, Sammy