search

OpenTalker / SadTalker

[CVPR 2023] SadTalker：Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation
https://sadtalker.github.io/
Other
12.04k stars 2.24k forks source link

Control head move and blink eyes? #176

Open merecesarchviz opened 1 year ago

merecesarchviz commented 1 year ago

it is possible to adjust this parameters?

because if we select remove head motion the character will not blink the eyes also! will be nice let the eyes blink

and about the head moving sometimes is to much! will be nice can control this

vinthony commented 1 year ago

Hi, we find eye blink/head motion coefficients will make an unstable paste back in full mode, so we remove them currently.

We will try to retrain the model for better performance.

james-s-tayler commented 1 year ago

I came here wondering the same thing. Sometimes with a character looking directly at the viewer it's possible to get an animation in full mode that includes blinking and doesn't move the head too much causing the picture to kind of having that tearing effect like it sometimes does, but when you disable the movement then the blinking also goes away.

SadTalker is pretty awesome, and glad to hear you're improving the model even further.

aruncambay commented 1 year ago

Hi, what does the blinking video expected to be? Should it be for the same image as the input image or any blinking video will do?

Inferencer commented 1 year ago

Hi, what does the blinking video expected to be? Should it be for the same image as the input image or any blinking video will do?

any blinking video will do

aruncambay commented 1 year ago

Thanks. I am getting an error while uploading the blinking video and trying to figure out what the reason could be. Should the duration of the blinking video match with that of the audio?

Inferencer commented 1 year ago

Thanks. I am getting an error while uploading the blinking video and trying to figure out what the reason could be. Should the duration of the blinking video match with that of the audio?

it should loop the blinking vid if shorter than audio, what error are you getting?

aruncambay commented 1 year ago

Hi, thanks for the reply, appreciate it! I am using the implementation on Replicate -- https://replicate.com/cjwbw/sadtalker and it says -'Path' object has no attribute 'split' when I try to upload any blinking video. Without that, it works fine. But, that's not useful for the application I am creating since it looks very unnatural without the blinking.

"