Closed dfulu closed 1 month ago
Just going through this to make sure I understand it. I thought I'd look at the SSIM between a random image from the leading channel and a blurry version to just see what happens. Top is the normal output of SSIM, bottom cuts out the border.
Here, the effect of NaNs appearing doesn't materialize. Do I need to be looking at all channels to see this working?
LGTM
So for documentation purposes, we had a discussion on this. Our tests were failing because of the very particular choice of putting a NaN in the top-left corner of our image:
There is some interaction here between this and setting gaussian_weights = False
. We avoid this here by consistently using gaussian_weights = True
everywhere, and fix the hyperparameters to match those in the original structural similarity paper. We didn't track down a concrete explanation of the all-NaN behaviour, but are satisfied that we now have a numerically stable output in the case of NaNs in the input!
Note: edited since first posted
By changing some of the parameters used in the
structural_similarity()
function fromskimage
, we can get the SSIM calculation to work with the examples in our tests which contain NaNs in the target image.In working on this, we discovered that the old configuration of
structural_similarity()
only fails if there is a NaN in the top left (i.e. aty[..., 0, 0]
) but doesn't fail if there are NaNs elsewhere. So we had only caught an edge case with our tests.However, I believe that this new configuration of
structural_similarity()
which passes with our current tests is still better because it is more robust and more referenceable. The new configuration ofstructural_similarity()
matches that used in the original paper . We use the hint in thestructural_similarity()
docstring to match to the original paper.This solves #11