search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
9.8k stars 2.13k forks source link

train wav2lip288 in a low loss,but it not perform well in inference! #604

Open killnice66 opened 6 months ago

killnice66 commented 6 months ago

i use hdtf for wav2lip288 training,nearly 170 0000 pictures ,16hours my syncnet eval loss is 0.3 and my L1 eval loss is currently 0.019051536196276822 and my sync eval loss is 0.18783933469512834 which seems good ,but my inference product the original frame with some blue dots on them ,what's the potential problem in it? image

my inference video (which is the same lip as original video) https://github.com/Rudrabha/Wav2Lip/assets/74774674/8e794215-a9b6-4c08-ac16-170f8fefef70

Ronincod3 commented 4 months ago

any updates?

LiuJiaqi0505 commented 2 months ago

i use hdtf for wav2lip288 training,nearly 170 0000 pictures ,16hours my syncnet eval loss is 0.3 and my L1 eval loss is currently 0.019051536196276822 and my sync eval loss is 0.18783933469512834 which seems good ,but my inference product the original frame with some blue dots on them ,what's the potential problem in it? image

my inference video (which is the same lip as original video) https://github.com/Rudrabha/Wav2Lip/assets/74774674/8e794215-a9b6-4c08-ac16-170f8fefef70

Hi br. The video looks like out of sync.