search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
10.3k stars 2.21k forks source link

Training Question #50

Closed qustions closed 4 years ago

qustions commented 4 years ago

I have done all the preprocessing and started the training but after 37 epoch Real: is 0 is this normal? or I have done something wrong 

Starting Epoch: 31
L1: 0.1545860916376114, Sync: 0.0, Percep: 0.6312605142593384 
| Fake: 0.75914174L1: 0.1545860916376114, Sync: 0.0, Percep: 0.6312605142593384
| Fake: 0.75914174L1: 0.15869291126728058, Sync: 0.0, Percep: 0.6291199922561646 
| Fake: 0.7615769L1: 0.15869291126728058, Sync: 0.0, Percep: 0.6291199922561646
|Fake: 0.7615769L1: 0.15976321697235107, Sync: 0.0, Percep: 0.624918540318807 
|Fake: 0.76644182L1: 0.15976321697235107, Sync: 0.0, Percep: 0.624918540318807 
|Fake: 0.76644182L1: 0.15976321697235107, Sync: 0.0, Percep: 0.624918540318807 
| Fake: 0.766441822052002, Real: 0.3562496453523636: : 3it [00:02,  1.07it/s]
Starting Epoch: 32

after 37 epoch Real: is 0

Starting Epoch: 37
L1: 0.1491541564464569, Sync: 0.0, Percep: 0.0
 | Fake: 27.63102149963379, Real: L1: 0.1491541564464569, Sync: 0.0, Percep: 0.0
 | Fake: 27.63102149963379, Real: L1: 0.15397701412439346, Sync: 0.0, Percep: 0.0
 | Fake: 27.63102149963379, Real:L1: 0.15397701412439346, Sync: 0.0, Percep: 0.0
 | Fake: 27.63102149963379, Real:L1: 0.15176475048065186, Sync: 0.0, Percep: 0.0
 | Fake: 27.63102086385091, Real:L1: 0.15176475048065186, Sync: 0.0, Percep: 0.0
 | Fake: 27.63102086385091, Real:L1: 0.15176475048065186, Sync: 0.0, Percep: 0.0 
| Fake: 27.63102086385091, Real: 0.0: : 3it [00:03,  1.04s/it]
Starting Epoch: 38
prajwalkr commented 4 years ago

Are you training with LRS2?

qustions commented 4 years ago

No on LRS3 even on LRS2 same after 999 steps

Saved checkpoint: train_checkpoint/checkpoint_step000003000.pth
Saved checkpoint: train_checkpoint/disc_checkpoint_step000003000.pth
Evaluating for 300 steps
L1: 0.026276566088199615, Sync: 0.0, Percep: 0.0 | Fake: 27.63102149963379, Real: 0.0: : 2it [00:03,  1.99s/it]

raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512, 1, 1])
prajwalkr commented 4 years ago

How many iters do you have per epoch? You should get a thousand iterations at least per epoch.

qustions commented 4 years ago

@prajwalkr I am getting 5 iters. How can I increase that iter is there a way to train only one video from the dataset and test it.

prajwalkr commented 4 years ago

You can increase the iter if you train on the whole LRS2 train set.

qustions commented 4 years ago

@prajwalkr @Rudrabha What is the difference between LRS2 and LRS3 I will modify the code but first I want to train on one person video and check the results what changes i need to do Things I know

  1. Fps 25
  2. preprocessing.py Code changes
  3. add folders of training and val in filelists Q: can i add full path example /home/user/lrs3/00001
prajwalkr commented 4 years ago

What is the difference between LRS2 and LRS3

https://github.com/Rudrabha/Wav2Lip/issues/49#issuecomment-691659887

will modify the code but first I want to train on one person video and check the results what changes i need to do

https://github.com/Rudrabha/Wav2Lip#training-on-datasets-other-than-lrs2

3. Q: can i add full path example /home/user/lrs3/00001

only if /home/user/lrs3/00001.mp4 is a video file