search

D-X-Y / AutoDL-Projects

Automated deep learning algorithms implemented in PyTorch.
MIT License
1.57k stars 283 forks source link

Why use exponential distribution to implement Gumbel softmax? #89

Closed pingguokiller closed 4 years ago

pingguokiller commented 4 years ago

In function select2withP of AutoDL-Projects\lib\models\shape_searchs\SoftSelect.py, the implementation of Gumbel softmax is below:

gumbels = -torch.emptylike(logits).exponential().log()

I don't understand why to use exponential distribution to implement Gumbel softmax?

D-X-Y commented 4 years ago

I follow the implementation of PyTorch at here: https://pytorch.org/docs/stable/_modules/torch/nn/functional.html#gumbel_softmax