probtorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration
http://pytorch.org
Other
3 stars 1 forks source link

Implemented Relaxed Distributions #113

Closed rachtsingh closed 6 years ago

rachtsingh commented 6 years ago

Not sure if this is the right name for this distribution (Concrete / GumbelSoftmax are other ideas), but this is what Tensorflow calls it. This PR uses the transforms machinery :)

I had to edit TransformedDistribution's log_prob method to take into account event_shape - this is probably not the right way to do it, but a quick first try that makes it work.

cc @fritzo

rachtsingh commented 6 years ago

Ok, made changes based on comments, and added the RelaxedBernoulli distribution. No longer blocked, so ready for another review if necessary @fritzo.

rachtsingh commented 6 years ago

Added tests that:

  1. Rounding the RelaxedBernoulli distribution gives the corresponding Bernoulli distribution
  2. Taking the argmax of the RelaxedOneHotCategorical gives the corresponding Categorical distribution
  3. As the temperature becomes very large, the first consistently gives 0.5 and the latter gives equal values for each index.