search

openai / pixel-cnn

Code for the paper "PixelCNN++: A PixelCNN Implementation with Discretized Logistic Mixture Likelihood and Other Modifications"
https://arxiv.org/abs/1701.05517
Other
1.91k stars 437 forks source link

How to understand the softmax sampling #24

Closed weixsong closed 6 years ago

weixsong commented 6 years ago

Hi,

In the code, logistic distribution sampling is done by:

    # sample mixture indicator from softmax
    sel = tf.one_hot(tf.argmax(logit_probs - tf.log(-tf.log(tf.random_uniform(
        logit_probs.get_shape(), minval=1e-5, maxval=1. - 1e-5))), 3), depth=nr_mix, dtype=tf.float32)

I could not understand how it it done this way.

Do I need to compute the CDF of pdf, and then for cdf, minus a random unifrom number, and take log option?

Why here minus a random uniform number make sense?

xmasotto commented 6 years ago

This is the Gumbel-max trick, used to efficiently sample from a softmax distribution: https://hips.seas.harvard.edu/blog/2013/04/06/the-gumbel-max-trick-for-discrete-distributions/

weixsong commented 6 years ago

@xmasotto , thanks very much. I got it.