search

TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.17k stars 273 forks source link

How manually control expression ?? #89

Closed yjhong89 closed 2 years ago

yjhong89 commented 2 years ago

Dear author.

Thank you for great work.

I am currently seeking a way how manually control expression (eye blink, mouth close... ) with FLAME parameters. I found that getting an eye blink by chaning FLAME's expression parameters in "edit_sequence.py", but I also wonder how we could get "blink_eye_beta"

Can you give me more detail how I can manually control expression with FLAME parameters as VOCA did an eye_blink ?

Thank you!

TimoBolkart commented 2 years ago

For the eye blink, we directly take the FLAME expression blendshapes (weighted by the eye blink parameters) and add them to the VOCA output mesh. This is possible as VOCA animates the mouth / lower face half, while the eye blink is in the upper part. We also show how to change the face identity by using FLAME's identity shape blendshapes. Generally editing the expressions is more complicated as they affect the mouth and therefore interfere with VOCA's generated mouth articulation.