search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
9.8k stars 2.13k forks source link

How to train your gan? #157

Closed wanghzhRun closed 3 years ago

wanghzhRun commented 3 years ago

why Percep loss keep increasing when using gan? image

prajwalkr commented 3 years ago

Your L1 and sync loss are quite low and decreasing rapidly. The GAN is hard to train due to the pose changes and artifacts that occur during training. We are working on an improvement for this at the moment.

wanghzhRun commented 3 years ago

your means is we should train gan without pretrain wav2lip model when we want to use gan to train wav2lip?

prajwalkr commented 3 years ago

No, I didn't mean that. I just told the reason why perceptual loss is increasing.