search

GaParmar / clean-fid

PyTorch - FID calculation with proper image resizing and quantization steps [CVPR 2022]
https://www.cs.cmu.edu/~clean-fid/
MIT License
903 stars 69 forks source link

resize function that can backpropagate gradient #6

Open betterze opened 2 years ago

betterze commented 2 years ago

Dear clean-fid group,

Thank you for sharing this great work, I really like it.

You mention in the paper that the PIL implementation is anti alias. But the PIL library could not back backpropagate gradient. Do you have plan to implement a way resize function that support anti alias and backpropagation? Or could you inform me the right way to do this?

I believe this will be very usefull for the community. For example, we invert a real image to the latent space of GAN, usually we dont use the full resolution image. The generated image must downsampling then compare to real image (LPIPS), and then backpropagate gradient. This is the implementation in Stylegan-ada (https://github.com/NVlabs/stylegan2-ada-pytorch/blob/main/projector.py#L97), it uses the torch resize function which could not anti alias as you mentioned in the paper.

Thank you for your help.

Best Wishes,

Alex

betterze commented 2 years ago

If I understand correctly, I need to blur the image first. The codes are not very easy from PIL (https://pillow.readthedocs.io/en/stable/_modules/PIL/Image.html#Image.resize).

GaParmar commented 2 years ago

We have not implemented resizing which can propagate gradient in this repository. There are some related libraries that might be of interest ResizeRight, kornia. Although we have not fully evaluated these for computing FID.