search

richzhang / PerceptualSimilarity

LPIPS metric. pip install lpips
https://richzhang.github.io/PerceptualSimilarity
BSD 2-Clause "Simplified" License
3.71k stars 500 forks source link

How to ignore some pixels when calculating the loss? #111

Open MargauxMasson opened 2 years ago

MargauxMasson commented 2 years ago

Hi,

I would like to ignore some pixels in the pairs of images I am comparing, how can I do this? I have the masks of the pixels I don't want to be used when calculating the loss.

Thank you!

richzhang commented 2 years ago

You can turn spatial=True https://github.com/richzhang/PerceptualSimilarity/blob/master/lpips/lpips.py#L23 and then multiply by the mask. Keep in mind that this won't mask the pixels out exactly, since the deep features have a receptive field. To truly mask out the pixels, you need to trace through the receptive fields of the deep activations like in https://openaccess.thecvf.com/content_cvpr_2017/papers/Gatys_Controlling_Perceptual_Factors_CVPR_2017_paper.pdf