Doubiiu / CodeTalker

[CVPR 2023] CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior
MIT License
537 stars 58 forks source link

Expressions for vocaset #4

Closed kmohanku closed 1 year ago

kmohanku commented 1 year ago

Hello,

Thanks for releasing this amazing work. I was curious if vocaset pre-trained models have diverse upper face expressions as well because the original dataset does not have them. If yes, could you comment on how you trained it?

Once again, great work! Thank you!

Doubiiu commented 1 year ago

Hi, thanks for your interest. I think there is no way for our model to produce diverse upper-face expressions like real humans, as the dataset itself lacks that. But we do observe that the facial animation produced by our method preserve more subtle details in the dataset, e.g, some high-frequency motions around the cheek and eyes, than previous methods. I think the essential way to improve that is training on more vivid datasets with diverse upper-face motions. And I believe, in this case, we can see the merit of the proposed method more clearly. Thanks again for your question~