search

nannau / ClimatExML

GNU General Public License v3.0
3 stars 1 forks source link

Choose one msssim package #1

Open sbeale007 opened 1 year ago

sbeale007 commented 1 year ago

There are multiple msssim metrics being used one in losses.py and one from torch metrics. Which one should be used. Window size was an issue for smaller fields.

nannau commented 1 year ago

From a design standpoint, it would be nice to keep all the metrics contained (i.e. only use torchmetrics).

The trouble is that MS-SSIM needs to be configured slightly differently depending on the size of the fields. I think the issue we encountered was that torchmetrics.functional.multiscale_structural_similarity_index_measure defaulted to a window size of 11 which was too large for the smaller fields (128x128).

i.e. we should use this function: https://torchmetrics.readthedocs.io/en/stable/image/multi_scale_structural_similarity.html. We should also configure the kernel_size in our config.yaml file. (this is the same thing as window size). The other parameters, like sigma, reduction, etc should be okay to leave at their default values.

So we should NOT use pytorch_msssim in losses.py just because it is more fiddly (i.e. I perform a rescaling of the data which is unnecessary for torchmetrics).

I think we should add something to the config.yaml file that allows us to easily configure this. This is a fairly simple task so I would encourage you to try and implement it and then submit a pull request!