search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
10.18k stars 2.19k forks source link

Correct Settings to reproduce same quality video in demo page #232

Closed onzone closed 3 years ago

onzone commented 3 years ago

First of all, thanks for this awesome project. I was trying to reproduce lip-synced video of same quality shown on the project page, but failed to do so. I believe there are few parameters that I can change i.e. pads, with/without gan and smoothening/no-smoothening. Are there any other parameters that I should change to improve the generated lip quality? I am attaching the generated videos(both with/without GAN) that I have got. I have downloaded input video(640* 480) and used following setting: no-smoothening: True pads: [0,20,0,0] Any help would be really great. thanks

https://user-images.githubusercontent.com/7061779/111430077-e0315f00-871f-11eb-9baf-376177715798.mp4

https://user-images.githubusercontent.com/7061779/111430084-e1628c00-871f-11eb-8931-79549c28ef36.mp4

onzone commented 3 years ago

Does it depend on wav2lip_batch_size also? Or is it used only for saving inference time?

Rudrabha commented 3 years ago

Please use different padding and different resize_factor values. The bottom padding of 10 to 20 is often required for better results. Also, check both the models. At times, the GAN one is inferior to the without GAN one and vice versa.