search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
10.18k stars 2.19k forks source link

Train our own models? - Stuck in Preparing Train stage... #280

Closed AlonDan closed 3 years ago

AlonDan commented 3 years ago

Hello, This is a known issue for unexperienced people for a while who's trying to understand HOW to train our own models. The explanations are very confusing and not focusing, no step-by-step what should we do exactly, what files we should put and where. there are some explanation about directories but it's not very clear.

THIS is how I tried to train my own model, I followed these steps:

1. First I've created the directories: "main" and lrs2_preprocessed on the ROOT of wav2lip (where all my ZIP project extracted) 2. I've renamed my VIDEO.MP4 to 5 digits, I wasn't sure where to put the 5 digits: 00001.mp4 file exactly so I put it inside both directories (just in case, again it's not very clear from the documents). 3. I tried to prepare for training following the first command from the main page, using: python preprocess.py --data_root data_root/main --preprocessed_root lrs2_preprocessed/

As soon as I tried this (ENTER) it start doing something in the background but then I got this:

cmd_2021-06-04_05-35-09

So I wasn't sure what's wrong (I'm not a programmer) I just tried to 'pip install' both 'Numba' and 'Jit" again, it re-installed it (already been installed from what it said) and tried to run the command above again. It stops, and I still get the exact same message...

That's where I'm stuck, I can't even start to train anything because of the first step isn't doing anything. Can anyone direct to a step-by-step (for none-programmers / newbies) to follow instructions of how to exactly train our own models in wav2lip please?

Either a VIDEO tutorial or a clear documented tutorial, maybe it's not complicated as it seems from the current main page? I'm not sure but it will be very helpful and appreciated by many others who wants to learn how to TRAIN their own higher quality models.

Thanks ahead for any help!

Rudrabha commented 3 years ago

Hello,

Apologies for the issues. Are you using the original LRS2 dataset or your own videos? I will try to help you by giving simpler instructions depending on this.

AlonDan commented 3 years ago

1. I don't know much about the LRS2 dataset quality, is it the current model we have with the low quality resolution? also, is it possible to train with this to get to higher resolution?

2. Yes I would like to train my own videos as well, I just have no idea how to do so.

Please try to explain it step-by-step for somebody who have no idea how to do this. This will sure help not just for me, but also to all the community. Your help is be very helpful and appreciated, thank you so much :)

Ziad-Mohamedd commented 3 years ago

@AlonDan Have you reached any progress? i am trying to do as you and i am stuck please reply asap

AlonDan commented 3 years ago

Unfortunately it's impossible to understand for newbies. At the moment I only use the pretrained extremely low resolution model, but the goal is to train my own high quality set.

Hopefully one day, somebody will make a step-by-step video tutorial on YouTube or something because there is no way to understand how to do it from scratch.