search

facebookresearch / meshtalk

Code for MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement
Other
368 stars 56 forks source link

How to train it on VOCA #16

Closed xiapengchng closed 2 years ago

xiapengchng commented 2 years ago

Thank you for your awesome work! I was wondering how to train it on VOCA. How to get the mask weight of upper face and lower face?

alexanderrichard commented 2 years ago

I have not trained the system on VOCA. It is not an ideal dataset because VOCA has almost no upper face motion and no eyelids etc. But of course you can run it. For the weight mask, we just loaded a template mesh into meshlab and selected all vertices that are part of the mouth manually. We assigned weight 1.0 to all those vertices and a lower weight to all other vertices based on their distance to mouth vertices in the neutral face expression (Gaussian decay based on distance).