Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
10.71k stars 2.29k forks source link

AVSpeech preprocessing #401

Closed lsw5835 closed 2 years ago

lsw5835 commented 2 years ago

Hi, If I change the fps to 25 as recommended by wav2lip model, the audio sync also changes. How do I get the audio sync right again?

The following code is used for changing fps. "ffmpeg -y -i {input_path} -r 25 {out_path} -hide_banner -loglevel error"

To change the sync of video, after I got the offset of the video using syncnet, I also used the following sync shifting code. But its output does not match the correct synchronization.

f"ffmpeg -y -i {input_path} -itsoffset {shift} -i {input_path} -ss {shift} -t {all duration of video - abs(shift)} -map 0:v -map 1:a {out_path} -hide_banner -loglevel error"

1105135335 commented 1 year ago

嗨,如果我按照 wav2lip 模型的建议将 fps 更改为 25,音频同步也会发生变化。 如何再次正确同步音频?

以下代码用于更改 fps。 “ffmpeg -y -i {input_path} -r 25 {out_path} -hide_banner -loglevel 错误”

要更改视频的同步,在使用 syncnet 获得视频的偏移量后,我还使用了以下同步移位代码。但它的输出与正确的同步不匹配。

f"ffmpeg -y -i {input_path} -itsoffset {shift} -i {input_path} -ss {shift} -t {视频的所有持续时间 - abs(shift)} -map 0:v -map 1:a {out_path } -hide_banner -loglevel 错误”

Have you found a way to sync?