search

TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.15k stars 273 forks source link

How to add or controll expression in the template animation? #34

Closed plmsmile closed 4 years ago

plmsmile commented 4 years ago

I want to add some expressions when the template is talking. But now I am not sure how to add it. Thanks for your advice~

TimoBolkart commented 4 years ago

We provide a demo (edit_sequences.py) to change facial shape for an animation. For this, please look into alter_sequence_shape within the edit_sequence.py line 150: model.betas[:300] = model_parms[frame_idx] The first 300 parameters of the models.betas account for shape changes, parameters 300 - 400 for expression changes accordingly. So if you change the model.betas[300:400] this will change the expression.

However, depending on the chosen expression, this might mess up the result as the expression and the speech articulation are not independent. Ideally one learns from emotional speech data how speech and emotions correlate.