search

TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.15k stars 273 forks source link

After add eyeblink, the face twitches #131

Closed hrWong closed 2 months ago

hrWong commented 3 months ago

Thank you for your outstanding work! When I work edit_sequences.py, there is a blink but there is a face jerk, before that the lip movement was correct, what could be the cause of this and what should I do to correct it.Here is video

https://github.com/user-attachments/assets/37cbc095-aef3-406d-960b-d017acfc632e

TimoBolkart commented 3 months ago

Can you please show the result before applying the edit_sequence script? Assuming that the lip sync was indeed fine prior to the edit_sequence, and given that the output sequence has eye blinks, the problem likely comes from the file parsing. How are your sequence frames named? Can you please check that the sorted filenames in the script indeed correspond to the order of the sequence frames?

hrWong commented 2 months ago

Hi @TimoBolkart, Thank you for your reply. I have fixed the issue. The problem was here, after I replaced the npy file with the predicted vertices, the results were normal.