The "Variation" parameter does not actually correspond to Boltzmann temperature

[samuela]

According the source code (https://github.com/distillpub/post--handwriting/blob/master/public/assets/model/model.js#L137) the variation parameter actually corresponds to applying to the temperature to the mixture probabilities, and multiplying all of the sigma_x and sigma_ys by the temperature. But this is not the same as adjusting a mixture of Gaussians by the temperature. This can easily be seen by considering a mixture of a single 1d Gaussian. The standard deviation of the updated distribution will be sigma * sqrt(T), not sigma * T.

[colah]

@hardmaru -- can you look at this?

[hardmaru]

That's a good point, and thanks for the catch. And in fact, in the Sketch-RNN paper, we have changed the definition to use sigma * sqrt(\tau), but have not updated the handwriting models accordingly. A pull request has been created to resolve this issue.

[samuela]

@hardmaru I'm not sure this quite solves the issue though. Unless I'm very mistaken

is not proportional to

I guess that it's a very sensible parameter to adjust and is easily understandable as the temperature of both the \pi_k and mixture component distributions. But it's not the same thing as the Boltzmann temperature of the full mixture distribution, as the footnote suggests.

[hardmaru]

In the footnote "Temperature is most commonly discussed in Boltzmann distributions, but can be generalized to all probability distributions", it is not implied that the temperature definition used here is the Boltzmann temperature for the full mixture distribution. We can consider rephrasing that sentence to avoid misunderstanding.