Training GAN with limited data

[a-ceron]

What

Create a pipeline using as reference Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T. (2020). Training generative adversarial networks with limited data. Advances in neural information processing systems, 33, 12104-12114.

Description

As in the Karras et. al. article, we should create a pipeline for the creationg of lenitucular efect over galaxy images

[a-ceron]

Resumen

Training generative adversarial networks using too little data typically leads to discriminator overfitin, causing training to diverge. Autors propose an adaptative discriminator augmentation mechanism that stabilizes trainin in limited data regiments. Witout change to loss functions or network architecture, and is applicable both when training from scratch and when fine-tunininf an existing GAN on another dataset.

Notes:

• Increasingly impressive results
• It remains challenging to collect a large enough set of images for a specific application
• Dataset augmentation is the standar solution against overfitting.
• GAN trained unde similar dataset augmentations learns to generate the augmented distribution.
• Use a range of augmentations to prevent the discriminator
• Good results are now possible using only a few thousand images.
• Frechet inception distance
• We start by studyng how te quantity of available training data affects GAN training
  • We approachthis by artificially subsetting larger dataset
• Training starts the same way in each case, but eventually the progress stops and FID starts to rise.
  • Conclusion: The less training data there is, the earlier this happens.

Questions

1. What did this papper? They are using and Adaptative Data Augmentation method/pipeline/classification to know if the training its better than other modelos that have a its database is bigger.
2. Which things are importatn to me? How they are put down the pipeline and they results about the errors with the normal data size and the results after using data augmentation. Also theris had a string indication of overfitting, the propose a way to tackel, prevent the discriminator from becoming overly confident.
3. What can I use of this papper. Some keys that help us to write some objectives like results (using our pipeline or use only GANS), Whis images are better to use

Lectures

1. Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196.

[a-ceron]

Scratch

Step 1: Finding some GANs models

• WGAN-GP [I]: Has been experimentally demostrated to be stable and robust against mode collapse.
• AC-GAN [I]
• BigGAN
• Progressive GAN
• FastGAN
• StyleGAN

Step 2: Train models with out using data

Transfering data: Transferring knowledge of pre.trained netowrks to new domains by means of fine-tunning is a widely used practice for applications based on discriminative models. Wang, Y., Wu, C., Herranz, L., Van de Weijer, J., Gonzalez-Garcia, A., & Raducanu, B. (2018). Transferring gans: generating images from limited data. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 218-234). This article evaluate several aspects of domain adaptation, icluding the impact of target domine size, the relative distance between source and targer domain, and the initialization of conditonal GANs. Using Knowledge from pre-trained nwtworks can shorten the convergence time and can significantly improce the quality of the generate images. That results also suggest that density is more important than diversity and a dataset with one or few densely samplesd classes is a better source model tha more diverse dataset

Step 3: Train models using data

• We generate images of (XXxXX) pixels, using standard values for hyperparameters. The source models are traindes with a batch of N images during I iterations using Adam and a learning rate of 1e-4. For fine tuning we use a batch size of N2 and a learning rate of 1e-4.
• Thw two netwotks of the GAN can be initialized with either random or pre-trained weights. Four possible combinations using a GAN

---

[a-ceron]

Transferring GANs: generating images from limited data

Resumen

Trasnferring knowledge of pre-trained networks to new domains by means of finetuning is a widely used practice for applications based on discriminative models. We evaluate several aspects of domain adaptation, including the impact of target domain size, the relative distance between source and target domain, and the initialization of conditional GANs. Using Knowledge from pre-trained networks can shorten the convergence time and can sifgnificatly improve the quality of the generated images. Our results also suggest that density is more important then diversity and a dataset with one or few densely sampled clasees is a better source model than more diverse datasets

Key points

• GANs can generate samples from complex image distributions
• The pretrained network is used to initialize the weights for a new task which are then fine tuned with the training images from the new domain.
• Evalluate the usage of pre-trained the quality of the generated images
• In general, a good metric shoud measure the quality and the diversity in the generated data. FID and WI are two metrics to comparate the models.
  • FID: Similarity between two sets. Using the full dataset with real images and a set of generated samples.
  • WI: As independen critic D. Approaching the Wasserstein distance between two datasets. Using a validation set and a set of generated images
• The two networks can be initialized with either random or pre-trained weights.
  • Transferring the discriminator is more critical than transferring the generator.
  • Trasnferring both obtains the best resultas. We also found that training is more stable in this setting. Training is more stable in this setting.
  • Networks adapted from a pre-trained model can generate images of given scores in signicantly fewer iterations.
  • GANs can generate images with good quality already with much fewer iterations.
• The number of training images is critical to obtain realistic images.
• Pre-trained networks can generate high quality images already in earlier iterations, in particular with sharper and more defined shapes and more realistic fine details.
• Adapting pre-trained networks can transfer relevant prior information
• the domain of the source model and its relation with the target domain are also a crtical factor.
  • We pre-trained GANs for the four source datasets and then trained five GANs for each of the four target datasets. Pre-traned GANs achive significantly better results

Some tips

• Visualisation of the metrics
• We evaluate our approac in a chanllengin setting where we use as fe as XXX images
• We computed FID for 250, 2500, 25000 iterations. We can observe how the difference between networks trained from scratch or from pre-trained weights is more significant for smaller samples sizes

[a-ceron]

Data section

Portal

https://portal.nersc.gov/project/cusp/ssl_galaxy_surveys/strong_lens_data/

Legacy Imaging Suerveys Data

https://www.legacysurvey.org/dr9/description/

Main paper: Mining for strong gravitational lenses with self-supervised learning

Information (aka data images) comes from the Dark Energy Spectroscopic Instrument Legacy Imaging Surveys' Data Release 9.

From this article, we got two data sets, the first one is a training set with 1615 and the second with 1195 new strong lens candidates (images that contain characteristics of strong gravitational lensing as viewed in the DESE Legacy Survey imaging.