search

Doubiiu / CodeTalker

[CVPR 2023] CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior
MIT License
515 stars 57 forks source link

Can't blink with pretrained model #31

Closed kendo6666 closed 1 year ago

kendo6666 commented 1 year ago

We use the pre-trained model to test on the vocaset dataset, but all the rendered videos cannot blink. How should we solve it?

Doubiiu commented 1 year ago

Hi, since the 4D face scans in VOCA and BIWI datasets don't contain blink motions, the model trained on them cannot produce facial animations with blink. You may need construct a new dataset (3d face reconstruction) or find a new dataset and train the model on it to generate facial animations with blink.

kendo6666 commented 1 year ago

@Doubiiu Thanks, I'll try, but it's not easy to find such a dataset