What model was used to extract the body pose ?

[ABH2024]

Hello ! Thank you very much for this open source work . I would like to know which model you used to extract the 104 joint angles for the body skeleton.
And is there is a way to view the skeletons only ? Thank you,

[evonneng]

Hi! Thanks for your interest in our work!

More about the 104 joint angles can be found by digging around the LBS model provided by: https://github.com/facebookresearch/audio2photoreal/blob/main/visualize/ca_body/utils/lbs.py

There's no direct code to extract it from say an image since this dataset was collected via mocap, but the pose definition can probably be found via the LinearBlendSkinning module in the above code.

And while I am not familiar with how to directly visualize the 3D skeletal points, you can get the untextured 3D mesh that the LBS generates via #26 . To do the 3D skeletal points, I believe you would have to fit a skeleton to the mesh and then visualize it.

Hope this bit of info helps!

[ABH2024]

Ok , Thank you ! I will dig into that. So the choice of 104 joint angles is only linked to the mocap technology that was used. By the way can you tell which technology did you use (if it is not a private information obviously ) ?

Also an other question in you paper in the Appendix B. Method, you talk about "SO(3) representation for the joint angles", is there any article or paper that can explain this kind of representation of the humain skeleton ?

Thank you in advance,

[evonneng]

Correct! The technology used is detailed here https://arxiv.org/abs/2105.10441. However it's something we cannot release fully. And absolutely! When referring to SO(3) I'm saying they are euler angles. I find this guide useful for understanding what that means: https://www.mathworks.com/help/nav/ref/so3.html. For the guide on how this is used for the human skeleton specifically, hope this article on wiki helps! https://en.wikipedia.org/wiki/Inverse_kinematics

[ABH2024]

Great, thank you very much ! :)