search

TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.14k stars 271 forks source link

Are vertices used in training regular vertices or transformed vertices? #98

Closed ulucsahin closed 2 years ago

TimoBolkart commented 2 years ago

Can you please specify what you refer to as regular or transformed vertices in this case? VOCA is trained on data that are registered to a common mesh topology (i.e., FLAME meshes), and all meshes are in "zero pose", which means effects of global rotation / orientation and head rotation around the neck are factored out. Does this answer your question?

ulucsahin commented 2 years ago

This answers my question, thank you.

On some works there are camera parameters and either perspective or orthogonal transformations are computed based on FLAME vertices. Now I understand that this is not used in VOCA and vertices are used without such transformations.