Architecture Clarification

[parhartanvir]

It's more of a doubt than an issue. Can you tell he why does the graph have two discriminators, viz. real_discriminator and fake_discriminator. I could not know from the paper or your blog post. Is this how you update wrights separately for images generated by generator and the real images? In that case are the real/fake inputs to the discriminator given at random or in a particular order?

By the way, thanks for the implementation! Tanvir

[Cristy94]

Yes, they are trained separately (eg. there are two discriminators, one for real, one from fake).

From the source code:

    with tf.name_scope("discriminator_loss"):
        # minimizing -tf.log will try to get inputs to 1
        # predict_real => 1
        # predict_fake => 0
        discrim_loss = tf.reduce_mean(-(tf.log(predict_real + EPS) + tf.log(1 - predict_fake + EPS)))

You can see how the final discriminator loss is obtained by combining the output of the two discriminators. The loss is with minus for the predict_real discriminator (eg. if it said it is fake but it was actually real it's not good).

Now, although the two discriminators are separate networks, they are trained using the same discrim_loss function, meaning that the output of one affects how the other is trained:

    with tf.name_scope("discriminator_train"):
        discrim_tvars = [var for var in tf.trainable_variables() if var.name.startswith("discriminator")]
        discrim_optim = tf.train.AdamOptimizer(a.lr, a.beta1)
        # See here that the gradients are computed for both discrminators using the same loss function
        discrim_grads_and_vars = discrim_optim.compute_gradients(discrim_loss, var_list=discrim_tvars)
        discrim_train = discrim_optim.apply_gradients(discrim_grads_and_vars)

PS: I just started learning about DL and using this library for a couple days, I might be wrong.

[WLpub]

I just start to read this code, and I think the point is:

with tf.variable_scope("discriminator", reuse=True):

two discriminators use and train same weights.