search

primepake / wav2lip_288x288

MIT License
560 stars 143 forks source link

about the PRelu & LeakyRelu improvement #50

Closed weizmann closed 1 year ago

weizmann commented 1 year ago

Thanks for your great work!

I have used the original wav2lip model to process wild video dubbing, and found there will be some abnormal color in the mouth occasional.

I think the reason is the original wav2lip model lacks proper softmax/relu process when handling color generation.

I read from your README and found that the 288x288 model used more powerful ReLU to process the convnet.

I have 2 questions about the improvement: Q1: Why did you choose to update the PRelu & LeakyRelu? can you give me some typical scenarios when PRelu/LeakyRelu works better? Q2: Will 288x288 model eliminates the abnormal color in the mouth?

Some bad cases in original Wav2Lip model:

image

image

image

weizmann commented 1 year ago

It will be better to give us some effect comparison videos ☺️

ghost commented 1 year ago

q1: I chosen these activations because I rearange of scale, [-1,1] instead of [0,1] by following: https://github.com/soumith/ganhacks q2: wav2lip 288x288 is just the upscale version of the original for the better resolution.