search

TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.17k stars 273 forks source link

Flame model pose explanation #92

Closed tearscoco closed 2 years ago

tearscoco commented 2 years ago

Hi,

Thanks for the amazing work open-sourced. I am manipulating the output mesh into flame params to know better how the flame model works by using compute_flame_param.py.

However, I am confused about the pose component of the flame model. I found the original pose to be shape size 12 (link here, and explanation as 「- articulation of neck (paramters: pose[0:3]), jaw (paramters: pose[3:6]), and eyeballs (paramters: pose[6:12])」) while the pose component from the flame model used in voca demo is 15 (where jaw movement is controled by [6:9]).

Could you please give a detailed explanation on each of pose params? Thanks !!

TimoBolkart commented 2 years ago

Hi, FLAME uses 15 parameters for pose, [0:3) global rotation around the root joint, [3:6) to rotate the head around the neck joint, [6:9) to rotate the jaw, [9:12) to rotate left eye, [12:15) to rotate right eye. The TF_FLAME separately considers the global rotation and prepends that afterwards to the 12-dimensional pose vector.

Does this answer your question?