search

ShmuelRonen / ComfyUI_wav2lip

A custom node for ComfyUI that allows you to perform lip-syncing on videos using the Wav2Lip model. It takes an input video and an audio file and generates a lip-synced output video.
89 stars 20 forks source link

Add arbitrary framerate as an input to node. #26

Closed FuouM closed 3 months ago

FuouM commented 3 months ago

Default is 30 as indicated in wav2lip_node.py mel_idx_multiplier = 80./30

ShmuelRonen commented 3 months ago

Thanks for your work. can you explain what the benefit of this change?

FuouM commented 3 months ago

Sometimes the video maybe 60fps or 25fps, which makes the wav2lip out of sync if you don't force the framerate of the input video to 30fps. The change does not affect any existing workflows and makes the node more robust.

ShmuelRonen commented 3 months ago

Sometimes the video maybe 60fps or 25fps, which makes the wav2lip out of sync if you don't force the framerate of the input video to 30fps. The change does not affect any existing workflows and makes the node more robust.

The original code should detect the frame rate automatically

FuouM commented 3 months ago

Actually not, as I've tried, mel_idx_multiplier = 80./30 this line splits the mel spectrogram to 30 indices, always. This is presumably the "framerate" at which the lipsyncing is done. Also, since the input images is just a sequence of images, they do not hold any fps information, so you would need the additional input for the fps.

ShmuelRonen commented 3 months ago

Actually not, as I've tried, mel_idx_multiplier = 80./30 this line splits the mel spectrogram to 30 indices, always. This is presumably the "framerate" at which the lipsyncing is done. Also, since the input images is just a sequence of images, they do not hold any fps information, so you would need the additional input for the fps.

Thanks. I will merge it!