Boese0601 / MagicDance

[ICML 2024] MagicPose(also known as MagicDance): Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion
https://boese0601.github.io/magicdance/
Other
615 stars 52 forks source link

the code of computing PSNR in the Disco repository is wrong #20

Open Delicious-Bitter-Melon opened 3 months ago

Delicious-Bitter-Melon commented 3 months ago

The Disco code of computing PSNR as follows is wrong: https://github.com/Wangt-CN/DisCo/blob/8538889c9ee9edd8dd43ffee182d1a91ce7a9828/tool/metrics/ssim_l1_lpips_psnr.py#L13.

image

As pointed out in https://github.com/Wangt-CN/DisCo/issues/86, the accurate code is mse = np.mean((original/1.0 - compressed/1.0) 2) instead of mse = np.mean((original - compressed) 2) , because original and compressed images are uint8 in their code, and (original - compressed) 2 will cause numerical overflow.

If you use their evaluation code of computing PSNR, please update your results.

Delicious-Bitter-Melon commented 3 months ago

In some cases, the wrong computation manner will increase PSNR but in some cases it decreases PSNR.

image

Delicious-Bitter-Melon commented 3 months ago

Besides, I am certain that the FID calculation method used by Disco is correct, but I am not sure if other metrics such as FVD are correct.