search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
10.28k stars 2.21k forks source link

Fake/Real stay at 0.69 #314

Closed ghost closed 2 years ago

ghost commented 3 years ago

I trained with multiple GPUs but Fake/Real just stayed at 0.69 4 GPUs with batch_size=128

L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 1it [05:02, 301.12s/L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 2it [05:02, 211.17s/L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 2it [05:02, 151.27s/it]

I don't know what the reason is?

zswzifir commented 2 years ago

I trained with multiple GPUs but Fake/Real just stayed at 0.69 4 GPUs with batch_size=128

L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 1it [05:02, 301.12s/L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 2it [05:02, 211.17s/L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 2it [05:02, 151.27s/it]

I don't know what the reason is?

Hi, I would wonder have you finally solved this problem？

windTwT commented 2 years ago

I trained with multiple GPUs but Fake/Real just stayed at 0.69 4 GPUs with batch_size=128

L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 1it [05:02, 301.12s/L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 2it [05:02, 211.17s/L1: 0.025484307669103146, Sync: 0.0, Percep: 0.6968241333961487 | Fake: 0.6908760070800781, Real: 0.6942079961299896: : 2it [05:02, 151.27s/it]

I don't know what the reason is?

hi! do you solve the problem?

hannarud commented 2 years ago

Hi @primepake! As you closed the issue, you probably found the solution to that? Could you share it, please?

ghost commented 2 years ago

you should check your dataset carefully, using syncnet python to filter your dataset

hannarud commented 2 years ago

Thank you for the response! What kind of problems should I check for? Video image & sound not synced? Or faces detected not everywhere?

ghost commented 2 years ago

you can check my repo for detail

https://github.com/primepake/wav2lip_288x288/issues/21

Utkarsh-shift commented 2 weeks ago

why is L1: 0.023011334964113256, Sync: 0.0, Percep: 0.9181626143044983 | Fake: 0.5949714519330208, Real: 0.5965797347869831: : 452it [01:38, 5.89iL1: 0.023011334964113256, Sync: 0.0, Percep: 0.9181626143044983 | Fake: 0.5949714519330208, Real: 0.5965797347869831: : 453it [01:38, 5.78i] the sync loss is always : 0.0