HumanML3D and MoMask- Data I/O query

[zybermonk]

Hello, Thank you for great works in the motion-language field.

I would like to get some clarity, in general when momask or any transformer model, while using HumanML3D dataset;

• Which numpy files are being used to train, and which data is obtained as the initial output from the models. Because when we setup HumanML3D dataset, we end up with new_joints and new_joint_vecs.
• I understand there are several conversions and transformation between them and they are obtained from the raw joints data, however, I have a confusion as to which of these data the model actually sees, and ultimately outputs as tokens.
• I assume from these output tokens the required formats (bvh, numpy, smpl etc) are recovered by processing further after the output from the models.

Please correct me if I am wrong, and kindly help me understand if it is the new_jointsor the new_joint_vecs that go into the model and come out along with their associated texts.

Thank you.

[Murrol]

Hi, thanks for you interests.

new_joint_vecs are motion feature vectors used as the model I/O and for better learning. Yes, the motion with traditional representation is recovered from this motion feature vector.

new_joints are 3D motion recovered from new_joint_vecs to examine the feature extraction and recovering functions during HumanML3D preparation.

[zybermonk]

Hi @Murrol , I would like to quickly follow up with some additional queries. Firstly thanks for the support.

• I learnt that the data (frames, joints, co-ordinates) is Y-up and in Right-Handed co-ordinate system, please correct me if Im wrong. However, I am unsure what rotation convention does this follow. Specifically the Euler angles/yaw,pitch,roll and their order + associated co-ordinates.
• I believe these rotations can also be obtained from the 263 vectors, could you please guide me which of these columns are needed to extract these and how (the original motion processing in HumanML3D is quite challenging to understand this)

I appreciate your assistance and thank you again in advance :)

[Murrol]

1. Yes, it's a y-up right-handed coordinate system. In SMPL pose parameter, the rotation representation is the Axis-Angle but not Euler Angle. You can convert the axis-angle to the Euler angle in your own rotation convention.
2. I'm not sure if we can extract the joint rotation information directly from the 263 vectors. For how to obtain rotations by optimization methods, you can refer to this answer https://github.com/EricGuo5513/momask-codes/issues/70#issuecomment-2316415620

[zybermonk]

One final question, so, the data (frames, joints, coordinates) is not in relative representation to its parent? So far, I have been assuming that's the case. Please confirm if it is

• global,
• local to pivot/root (i.e., component),
• or local to parent (relative)

Thank you for the answers. I will try the suggested method to extract angles.

[Murrol]

they are global positions