Validate a GAN

[dantp-ai]

What is the best way to validate a GAN, since we can't straightforward track one loss. E.g. in case of SISR, is the choice of best PSNR a reasonable act?

[vinthony]

I think the most meaningful way to validate is the subject score.

GAN always generate "good" image but poorly in the evaluation.

pix2pix utilize subjective experiments and FCN-score on segmentation for validate.

In SRGAN, validate score of GAN, based on SSIM and PSNR, is a bit worse than the method without GAN.

I had tried to use GAN in some other tasks, the regress loss of GAN is also worse than the method without GAN, which is also experiments in pix2pix.

As far as I know, it seems that all the numerical evaluation have poor scores in GAN although GAN based methods have the better appearance.

[rafaelvalle]

A note on the evaluation of generative models Do GANs actually learn the distribution? An empirical study

[piyush01123]

I am having trouble making my generator converge. After some 500 steps of training, My discriminator reaches near 100% accuracy and my generator's accuracy drop to 5%. My code is here https://github.com/piyush-kgp/GAN-Lab/blob/master/dcgan/celeb_dcgan_hd.py

I am relatively new to GANs and would appreciate if someone could help me out. Thanks very much.

[R1j1t]

I read this in a retweet from Ian Goodfellow that "GAN's may be evaluated based on how smooth (disentangled) the latent space interpolations are." This seems to me a good evaluation metric as well, maybe qualitatively but not quantitatively. Here is the tweet

[miqbal23]

I have an almost similar question to this. Say you train a GAN in 100 epoch. Since GAN is unstable (in most case), how do you evaluate and pick a best-performing checkpoints gathered from the training process?