Closed xz6014 closed 5 years ago
It is a hard one-hot sample generated at each iteration per GPU (If you use multiple GPUs). Our implementation is similar to that of the hard mode of torch.nn.functional.gumbel_softmax(). Please note that torch.nn.functional.gumbel_softmax() has bugs, so please be careful if you are using that function. See here https://github.com/pytorch/pytorch/pull/20179
Thank you very much for your clarification. It's greatly appreciated.
Hi there,
Just wondering how was the gumbel-softmax implemented in your scripts. Was a hard one-hot sample generated at each iteration of the searching phase, or a soft one? I noticed there's an option in the torch.nn.functional.gumbel_softmax() of setting 'hard' to true to generate a one-hot sample. Was this the approach that you took?
Many thanks,
X