Using pig.fsim for images with more than 3 channels

[pooya-mohammadi]

When I apply pig.fsim to images with more than 3 channels, I get the following error:

RuntimeError: Given groups=1, weight of size [2, 1, 3, 3], expected input[1, 32, 192, 256] to have 1 channels, but got 32 channels instead

The code results in the above-mentioned error:

import torch
import piq
from skimage.io import imread

@torch.no_grad()
def main():
    x = torch.tensor(imread('../tests/assets/i01_01_5.bmp')).permute(2, 0, 1)[None, ...] / 255.
    y = torch.tensor(imread('../tests/assets/I01.BMP')).permute(2, 0, 1)[None, ...] / 255.

    if torch.cuda.is_available():
        x = x.cuda()
        y = y.cuda()
    x = x[:, :2, ...].repeat(1, 16, 1, 1)
    y = y[:, :2, ...].repeat(1, 16, 1, 1)

    fsim_index: torch.Tensor = piq.fsim(x, y, data_range=1.0, reduction='mean', chromatic=False, scales=8)
    fsim_loss = piq.FSIMLoss(data_range=1., reduction='none', chromatic=False)(x, y)
    print(f"FSIM index: {fsim_index.item():0.4f}, loss: {fsim_loss.item():0.4f}")

if __name__ == '__main__':
    main()

Is it possible to apply fsim to images with more than 3 channels? If it is possible, am I making a mistake or the feature is not implemented yet?

[denproc]

Hi @pooya-mohammadi,

Thank you for raising the issue with a case, when images with more than 3 channels are processed. Our implementation follows the original FSIM methods that is computed for 1-channel luminance component of images. In practice, the greyscale images are processed as is. In case of RGB images, we transfer the images into YIQ colour space and use Y component to deliver FSIM. Unfortunately, it means that images with different number of channels or colour schemes cannot be processed by design. However, if there is a way to retrieve luminance component from your n-channel input, you can use this component for FSIM computation using piq.fsim.

[denproc]

I'm closing the issue. In case of any additional feedback, feel free to re-open the issue.