search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
10.28k stars 2.21k forks source link

about training samples #375

Closed T5eng closed 2 years ago

T5eng commented 2 years ago

I tried to fine-tune the pre-train model with official LRS2 dataset and my own dataset.

Looks like what the model learned at training phase is simply copying the lower half face to the generated image. The first 2 images are sampled from LRS2, and the last from my own dataset. from left to right are reference, input, generated and gt frames image image image

trivikramak commented 2 years ago

Hi, Could you get access to the LRS2 dataset? Also, how did you pre-process your own dataset?

T5eng commented 2 years ago

apply for it at the official website. pre-process code is included in wav2lip source code. :)

hnsywangxin commented 5 days ago

@T5eng I have the same problem, can you tell me how to solve it. thank you