I would like to have the hand model after convex decomposition. Also, could you please provide some hints on Offset Regression?

[MY-CODE-1981]

train_multiview_loader_model.zip

I have some concerns regarding the following chapter of the paper:

7. More Implementation Details In the convex decomposition of hands, the vertex numbers of 20 local meshes are 45, 61, 43, 45, 92, 34, 41, 62, 44, 44, 58, 42, 40, 60, 41, 35, 64, 28, 50, 62, respectively (Sum is 991).

Does this mean that it is necessary to perform a convex decomposition of the MANO hand mesh in advance using Blender or another tool, and then externally record the vertices, faces, and indices? I am struggling with the explanation of how the 991 dimensions are reduced to 778 dimensions by allowing mesh overlaps after translating and rotating each convex part. If possible, could you provide me with an OBJ file or Blender file where the hand model after convex decomposition is layered by parts? Any file that keeps the meshes with the numbers 45, 61, 43, 45, 92, 34, 41, 62, 44, 44, 58, 42, 40, 60, 41, 35, 64, 28, 50, 62 in separate layers would be helpful. Additionally, I am currently rewriting the stage 1 training part according to the zip file, but I have only been able to complete the Offset Regression, and I would appreciate any hints on coding the subsequent processes.

[jackyyang9]

Hi, thanks for your interests ! Our code will be open-source in recent !

In the convex decomposition, I manually decompose the MANO mesh template to obtain 20 local meshes. Then I obtain the vertex order of each mesh according the vertex coordinates in template, such to obtain decompostion matrix ${\bf M} \in {\bf R}^{991 \times 778}$. Since the existance of overlap parts between neighbouring local meshes, 20 local meshes have 991 vertices in all.

Note that the decompostion matrix in this way is suitable for all MANO meshes, since they have same topology relationship. To obtain the merging matrix, you can simply compute the Left Inverse Matrix of ${\bf M}$.

Best regards! By the way, welcome to star our code : ) and feel free to contact me via email: jianyang0227 at gmail dot com

[MY-CODE-1981]

Thank you very much for your detailed response and for providing clarification regarding the convex decomposition process in MLPHand. Your explanation about the decomposition matrix and the use of the Left Inverse Matrix to merge the overlapping parts was very helpful.

After reviewing your response, I have a follow-up question regarding the MANO hand model. Since the MANO hand model changes its shape with different poses, I am curious about the importance of focusing on mesh IDs. If the hand's topology remains unchanged, I can see how a single convex decomposition can be reused across various poses. However, could you clarify further how this approach ensures that the convex decomposition is valid for all poses?

Additionally, I would like to confirm that performing the convex decomposition only once is sufficient, as long as the mesh IDs are consistent. In this case, do you recommend any specific guidelines for handling mesh overlaps when the hand mesh significantly deforms due to changes in pose?

Additionally, I was wondering if it would be possible for you to provide a list of the fixed topology vertex IDs after the convex decomposition.

[jackyyang9]

Hi, sorry for my late reply.

I have shared the convex decomposition matrix ( 991x778 ) in our code, please see Diag_Vertex.pth. The non-zero value in each row indicates a vertex ID. You can easily implement the convex decomposition for any MANO mesh via doing left multiplication. Our convex decompostion works like the joint regression matrix (21x778 ) in MANO model.

For the overlap, our decomposition is lossless in data preprocessing.

Thanks for your interests again ! Our code will be totally open-source in next few weeks.