search

ZiqiaoPeng / SyncTalk

[CVPR 2024] This is the official source for our paper "SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis"
https://ziqiaopeng.github.io/synctalk/
Other
1.34k stars 161 forks source link

Background appearing on output #208

Open thatnerdyaigirl opened 3 months ago

thatnerdyaigirl commented 3 months ago

Hi, I tried to finetune it as well with hubert and ave but I keep getting these background dots of the background colour, how can I resolve this? image

thatnerdyaigirl commented 3 months ago

Hey @ZiqiaoPeng , this issue is at fixed time frames, no matter if i train with hubert or ave or give any inference audio to test. Tried to train with higher num_rays and patch size but still the same issue

varunjain99 commented 3 months ago

I'm guessing if it happens with any inference audio, the issue is from the preprocessing (e.g. face mask generation) of that particular frame. Maybe check the face mask?