search

jorge-pessoa / pytorch-msssim

PyTorch differentiable Multi-Scale Structural Similarity (MS-SSIM) loss
Other
449 stars 69 forks source link

Big in SSIM implementation, don't use this code for perceptual quality estimation #22

Open zakajd opened 3 years ago

zakajd commented 3 years ago

Hi This code contains the same error as skimage, you can read full description here: https://github.com/scikit-image/scikit-image/issues/5192

Shortly, when used for estimation of perceptual quality, authors of original paper proposed to downsample images first to make SSIM focus on major differences between reference and distorted inputs.

So what? If you are using this implementation as a loss function for CNN, you're likely leading it in the wrong direction.

Alternatives You can find correct implementation of SSIM, MS-SSIM and some other metrics here: https://github.com/photosynthesis-team/piq

jorge-pessoa commented 3 years ago

Thank you for raising this issue. I will look into adding an optional flag that applies the correct average pooling before the SSIM computation. Note that this is also an issue on the original SSIM from which this repository was forked so you may want to raise this issue there as well.

I will leave the issue open until this is resolved in master.

charleneolive commented 3 years ago

I don't think it's an error per say, it's not very explicitly mentioned in the paper either, although during the experiments the authors did mention that they downsampled the images. In addition there are two implementations by the author here: http://www.cns.nyu.edu/~lcv/ssim/, with the second implementation similar to the first but with an additional preprocessing step. Nevertheless, I think it's good to take note of this for any perceptual quality tasks:)

zakajd commented 3 years ago

While it's not an error, usage of SSIM without preprocessing leads to a significantly worse performance as a "perceptual quality estimator" which is 90% of its use cases nowadays.