search

photosynthesis-team / piq

Measures and metrics for image2image tasks. PyTorch.
Apache License 2.0
1.4k stars 120 forks source link

SSIM non-rectangular kernel size #306

Open Willtl opened 2 years ago

Willtl commented 2 years ago

Is it possible to use SSIM for sequences without reshaping them? For example, through a [1, 1, 1, 11] kernel size instead of a [1, 1, 11, 11]?

The reason I’m asking is that I’m doing self-supervised learning on sequences using 1D CNNs. Surprisingly, when I use SSIM instead of L2, even though I have to reshape the sequences to rectangles (the length is a perfect square number), it gives me better results. I wonder if it would be possible to apply SSIM (without the requirement of reshaping the sequences) through the use of a non-square window.

In this case, parameter kernel_size could take a tuple of two ints – in which case, the first int is used for the height dimension and the second int for the width dimension

zakajd commented 2 years ago

Hi @Willtl thanks for a proposal! Yes, your idea seems valid. I'll discuss this refactoring with development team and respond later this week 👍🏼

Alternative solution will be to pass kernels as matrixes to functional interface of SSIM and other metrics. This will also solve #227 and avoid creation of kernels in class interfaces each time. But this kind of refactoring requires much more effort and can't be done quickly

Willtl commented 2 years ago

@zakajd is there any chance I can test my code with the non-rectangular kernel before the end of this month? I have a deadline for submission on the 30th of March.

zakajd commented 2 years ago

@Willtl Me and other guys are currently busy with other PRs, so can’t promise that.

You can fork and implement this by yourself by modifying gaussian kernel function and opening PR