search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
9.8k stars 2.13k forks source link

Has anyone looked at super resolution to improve the quality of output videos? #178

Closed Shriram-Coder99 closed 3 years ago

Shriram-Coder99 commented 3 years ago

Since the output videos are blurry, it is possible to apply GAN based super resolution on the output frames to improve the quality.

For example: https://github.com/ewrfcas/Face-Super-Resolution

I want to ask if anybody has explored other options for super resolution with better results.

Before super resolution, i.e. output from Wav2Lip

https://user-images.githubusercontent.com/56973022/103260774-ee939880-49c4-11eb-81fc-7f907a78b783.mp4

After super resolution:

https://user-images.githubusercontent.com/56973022/103260822-171b9280-49c5-11eb-81ee-2c00809d546a.mp4

viet-tin-tech commented 3 years ago

I had quite bad result with high res video, can you share your work.

prajwalkr commented 3 years ago

We had tried it long back, but it led to poor results. We are working on directly training on higher resolution data.

asdqe5 commented 3 years ago

@Shriram-Coder99 Can you share your work? I just keep trying to get high quality video. but I couldn't get any good results. Please help me.

FrischJulien commented 3 years ago

@prajwalkr May I ask you which dataset did you use?