search

Doubiiu / CodeTalker

[CVPR 2023] CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior
MIT License
538 stars 58 forks source link

Different training results #42

Open mhxlearn opened 1 year ago

mhxlearn commented 1 year ago

I completed two stages of training using the vocaset dataset, but when I input the audio test, the output results are not good, except for the first few frames, the later face is not much movement.

Doubiiu commented 1 year ago

Hi. I am not sure about the reason either. Can you check the results of early model checkpoints? Or even have another try?

mhxlearn commented 1 year ago

Hi. I am not sure about the reason either. Can you check the results of early model checkpoints? Or even have another try?

ok, i'll try again and hopefully keep this question open for now

Shirley-0708 commented 1 year ago

https://github.com/Doubiiu/CodeTalker/issues/42#issue-1807406056 The same.

HiouKaoru commented 6 months ago

Hi, I have been working with the dataset and came across a small issue in the subj_seq_to_idx.pkl file. It seems that the entries for "FaceTalk_170811_03274_TA" and "FaceTalk_170809_00138_TA" have missing sequence indices. Specifically, "FaceTalk_170811_03274_TA" is missing sequence index "sentence24" and "FaceTalk_170809_00138_TA" is missing sequence index "sentence32".

e.g. FaceTalk_170811_03274_TA : Missing sentence24 242096831-14337bbd-fb75-4264-b91d-12afff8d2696

That is causing me to not be able to train correctly and I was wondering if you have encountered this problem and how you solved it.