Question - no MINIST labels passed in training?

[jandom]

Hi there,

@bstriner this is a great package with powerful and comprehensive examples! Really enjoy working with it!

Quick question, something that I tried to understand from the source, looking at the simplest gan example in examples/example_gan.py

    xtrain, xtest = mnist_data()
    y = targets(xtrain.shape[0])
    ytest = targets(xtest.shape[0])

xtrain and xtest are what I expect them to be (-1,28,28) batches of data However y and ytest are just tuples (of length 4) filled with 0 or 1 by gan_targets -- I was expecting those to be the true MINIST labels?

Sort of like in this example https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py

(x_train, y_train), (x_test, y_test) = mnist.load_data()

Maybe this is just me not getting something here about GANs, or maybe other users have similar questions.

[bstriner]

@jandom Typical feed-forward networks are discriminative models. They predict Y given X. P(Y|X).

A GAN is a generative model. It is trying to understand some structure of the X data without knowing anything about Y. It is modeling P(X). Think of it in the same way that autoencoders try to extract some sort of structure.

There are related models. For example, CatGAN predicts X based on Y. P(X|Y). BiGAN tries to infer some hidden representation P(Z|X), P(X|Z).

If you are just trying to guess labels, discriminative models are fine. If you're trying to understand the internal structure of how that data is generated, you need a generative model. Discriminative models are easier but generative models can be much more powerful.

You can also use generative models to do some level of preprocessing. In the same way that you can stack autoencoders, you could use a BiGAN to generate some hidden representation Z of X, and then use a discriminative network to predict Y based on Z.

So, there are a lot of variations of GANs depending on what you're trying to model, and they can be combined with traditional discriminative models to do whatever it is you're trying to do.

The labels in the context of the example GAN are the target for the discriminator. The discriminator tries to maximize real data and minimize fake data. The generator then tries to adapt to the discriminator.

Cheers

[mynameisvinn]

GANs are not meant to classify digits ("is this the number 1 or 2 or 3...?"). they are meant to distinguish real/fake (that is, a binary classifier).

[jandom]

many thanks, both – those are excellent answers! when it comes to brevity, they fall on the opposite ends of the spectrum :)

asked this question many months ago, where my knowledge of the field was scarce and confused (it still is, maybe only slightly less so :))