junyanz / pytorch-CycleGAN-and-pix2pix

Image-to-Image Translation in PyTorch
Other
23.09k stars 6.32k forks source link

DiffAugment in an image translation task while maintaining explainability #1521

Open aarontyliu opened 1 year ago

aarontyliu commented 1 year ago

Hi, I am wondering how to use DiffAugment in an image translation task while maintaining explainability of the feature maps in the generator?

I am trying to apply DiffAugment with the policy of 'color, translation, cutout' with CUT and CycleGAN models in my research project. I am still exploring CycleGAN, but, with CUT, I observe the augmentation can delay domination of the discriminator while obtaining lower FID when other hyperparameters are left unchanged. However, the feature maps in the generator get "augmented". They become superimposed of multiple shifted maps, which might reduce the potential of other downstream tasks of interest that require explainability. I wonder if there's any suggestion that you can provide.

Thank you for creating these methods : D