search

junyanz / pytorch-CycleGAN-and-pix2pix

Image-to-Image Translation in PyTorch
Other
22.8k stars 6.29k forks source link

Is random cropping helpful for generative models? #892

Closed ThisIsIsaac closed 4 years ago

ThisIsIsaac commented 4 years ago

I understand that random cropping is a technique often used in classification networks, and for good reasons. However, is it also helpful for generative models? If so, it would be great if you could refer a paper for me to read on a little more of the effect of random cropping on generative models.

Thanks

junyanz commented 4 years ago

I don't have a paper in my mind. But it is a common technique for many neural networks. In our experiments, we observed that it could prevent overfitting, especially for small datasets.

ThisIsIsaac commented 4 years ago

@junyanz Wow, thanks for your prompt reply :)

But it is a common technique for many neural networks.

I don't mean to make this in to a prolonged QnA session, but I don't intuitively understand the benefit of cropping for generative tasks.

For classification, a few intuitive explanations are:

  1. prevents overfitting to a set of specific features by randomly hiding some features in the image
  2. Prevents the network from paying attention to noise (ex. background ) instead of the object of concern (ex. cat & dog)

But how does this generalize to generation tasks?

junyanz commented 4 years ago

For image-to-image translation, 1 still holds. For some datasets with only a few hundred images, it is quite easy for the generator to memorize the training set. But it will not generalize well for the test set. For some tasks such as edges2cats, you want to the input cat edge to appear in many locations, so that the model can work for user drawing at different locations at test time.

zjuxiaobaiq commented 4 years ago

For image-to-image translation, 1 still holds. For some datasets with only a few hundred images, it is quite easy for the generator to memorize the training set. But it will not generalize well for the test set. For some tasks such as edges2cats, you want to the input cat edge to appear in many locations, so that the model can work for user drawing at different locations at test time. i wonder that whether the role of resizing before random cropping is just for data augmentation ? Could you tell me what does the operation resizing before random cropping really do?

junyanz commented 4 years ago

Resizing is not part of data augmentation. Sometimes, if your dataset has images with different sizes, you want to make sure they are the same size.