I want to ask you about the idea of the article, that is, why you should use Gumbel-softmax - if I understand correctly, the input in this article is not sampling.Even with softmax, there should be no differentiable problems without reparameterization.
From the perspective of solving argmax‘s problem, I think softmax with temperature τ should also achieve a similar effect.
My understanding is that according to the statement in Alg1 and 2, a one-hot control variable needs to be output, but the specific explanation is still vague.
ithok
commented
2 years ago
Hi author, thanks for your work!
I want to ask you about the idea of the article, that is, why you should use Gumbel-softmax - if I understand correctly, the input in this article is not sampling.Even with softmax, there should be no differentiable problems without reparameterization.
From the perspective of solving argmax‘s problem, I think softmax with temperature τ should also achieve a similar effect.
My understanding is that according to the statement in Alg1 and 2, a one-hot control variable needs to be output, but the specific explanation is still vague.
Best wishes！