Very impressive model much better than wav2lip. One Caveat is prone to some glitchiness do you have some suggestions for how to overcome?

OpenTalker / video-retalking

[SIGGRAPH Asia 2022] VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild

https://opentalker.github.io/video-retalking/

Apache License 2.0

6.44k stars 953 forks source link

Very impressive model much better than wav2lip. One Caveat is prone to some glitchiness do you have some suggestions for how to overcome? #155

Open JieLuChen opened 11 months ago

JieLuChen commented 11 months ago

Hi I have noticed that there is some glitchiness around the mouth area and this also happens when the reference video has no speech but has poses and silence. Do you have recommendations for how to smooth this out? I don't think the enhancers help here and this seems more like a landmark issue. What are your thoughts?

AIhasArrived commented 10 months ago

Is it very slow for you aswell? I did not use wav2lip is it as slow? Another question @JieLuChen , what are all the tools you have used so far? I would like to try them all.