Question about gradient of transforms

[chkimmmmm]

https://github.com/rinongal/StyleGAN-nada/blob/775b8d108be92e49fdc42d75dfdcf7f16255a59f/ZSSGAN/criteria/clip_loss.py#L41

Hello, thank you for sharing great work. I have a quick question about this line. To efficiently training G_train, I thought you might use gradient of CLIP. From the line I referenced, you used torchvision's transforms, which is not passing gradient. In this case, G_train could not get gradient from CLIP. Without gradient of CLIP, G_train is trainable as we want?

Thank you. Best, chkimmmmm

[rinongal]

Hi,

We are using the gradients from CLIP. It's the only source of a loss, and the network wouldn't change at all if this was not the case.

Some torchvision transforms do indeed pass gradients. You can try this yourself, or check the warnings on official documentation like the resize transform which explicitly list arguments that will cause autodiff to stop functioning. We do not use any transforms that would require us to move to PIL and back, or any anti-aliasing steps, so gradients do pass through.

Just to be extra sure, I verified things with torch.autograd.gradcheck() and the numerical gradients do match the analytic ones.

[chkimmmmm]

Thanks a lot! I will check using the function you shared.