search

earthnet2021 / earthnet-toolkit

Other
11 stars 4 forks source link

SSIM calculation #3

Open nikoskot opened 2 days ago

nikoskot commented 2 days ago

I believe there is an issue with the calculation of the SSIM metric inside 'parallel_score.py'. I can see that it uses the metrics.structural_similarity() function from skimage. As from the end of 2022 this method needs the 'data_range' parameter to be explicitly set by the user, otherwise it throws an error.

The real issue though is that before the assignment of the parameter became mandatory, if you ignored it, the value would be calculated automatically and it would be set to 2 for floating point inputs, like in the case of Earthnet2021 images. This is also explained in the official skimage documentation, where it is also mentioned why it should be 1 instead of 2 (images should not have negative pixel values). Using data_range = 2 also yields a higher SSIM metric, compared to when setting it to 1.

vitusbenson commented 2 days ago

Oh wow! Great catch @nikoskot ! Feel free to open a pull request :) But please note also, we have moved away from using EarthNetScore, see the recent GreenEarthNet paper https://github.com/vitusbenson/greenearthnet?tab=readme-ov-file