search

lixiny / manotorch

MANO in pyTORCH (anatomical constraints, anchors, etc)
GNU General Public License v3.0
166 stars 11 forks source link

Minor questions about AxisLayerFK #5

Closed walsvid closed 1 year ago

walsvid commented 1 year ago

Hi @lixiny , thanks for your great work! I have some minor questions about AxisLayerFK.

  1. Could you please explain the abbr. in axislayer.py (e.g. TMPL_R_p_a)? What is the full namep and a.

  2. The thumb of the AxisAdaptiveLayer initially selected (1,1,1) as the basis of manual selection for subsequent orthogonalization. Does this choice affect the overall local basis of the final Anatomy?

  3. I noticed that the plausibility of AxisLayerFK output depends on the local coordinate frames of AxisAdaptiveLayer. AxisAdaptiveLayer is constructed based on pose==0, shape==0. Does the shape affect the positional relationship of MANO's fingers in the rest pose? For example, very extreme big hands or small hands. Will the gap between the basis obtained by initializing with shape=0 and the actual anatomy basis(with shape!=0) affect the rationality of the local euler angle output by AxisLayerFK?

lixiny commented 1 year ago

Thanks for your interest in our work!

  1. The MANO model has provide a set of coordinate basis. Our contribution is to define a transformation from the MANO predefined local basis to the local basis corresponding to anatomical relations. This transformation is independent of the pose and is therefore solved only once, for the zero pose and mean shape. It can then be used as a fixed value, denoted as TMPL_R_p_a in the AxisLayerFK module (where "p" is an abbreviation for "predefined" and "a" is an abbreviation for "anatomy"). With this transformation, we can decompose the rotation of the child in relation to the parent in any pose into the rotations of [the child in relation to the anatomy frame] and of [the anatomy frame in relation to the parent's predefined frame] (constant). We supervise the rotation of [the child in relation to the anatomy frame] to prevent abnormal twisting rotations (e.g., rotations around the (1,0,0) twist axis).

  2. The direction of the upper axis of the thumb, (1,1,1), was determined using empirical knowledge and may not align precisely with anatomy. However, the design of our loss mechanism was intended to serve as a complementary constraint during the process of optimizing pose, and this design is satisfactory for our needs.

  3. We acknowledge that the basis obtained through this method may not be appropriate for extreme shapes. As an intuitive workaround, we propose modifying the initialization function of the AxisLayerFK module to replace the tmpl_shape parameter with the shape obtained from network prediction or ground truth, while retaining all other components of the function.

Lixin.