liutaocode / DiffDub

[ICASSP 2024] DiffDub: Person-generic visual dubbing using inpainting renderer with diffusion auto-encoder
https://liutaocode.github.io/DiffDub/
Apache License 2.0
47 stars 8 forks source link

Training stage 2 and evaluation code #6

Open MatthieuFP opened 1 month ago

MatthieuFP commented 1 month ago

Hi,

Thanks a ton for your work and having open-source almost everything! Would it be possible to release the stage 2 training code of your method + the evaluation code please? I've been trying to reproduce your results but I face some problems in both steps to match the results (due to some details I may have got wrong).

In case it's not possible for the moment, I would have some questions: 1/ Am I right saying you're training and evaluating on the 10 first seconds of each video? 2/ Is the evaluation scores for image quality only computed on the lower part of the face (using landmarks previously extracted)? 3/ Is it possible to have the list of training, dev, test splits you used please (sorry in advance if I missed them somewhere)?

Thanks a lot again!