Open agadetsky opened 4 years ago
I was looking to see if this was implemented, but apparently not. I did find a project that implements PL distribution in Torch, so I will try that. https://github.com/agadetsky/pytorch-pl-variance-reduction
We have an implementation of the Plackett-Luce distribution supporting batching with permutations of varying length. I'd be happy to contribute it to pytorch if there is interest.
🚀 Feature
Add PlackettLuce and RelaxedPlackettLuce distributions. It is a simple distribution over permutations.
Motivation
For optimization over categorical/binary variables (i.e. variational inference with discrete variational approximate posteriors, RL and etc) we already can use
RelaxedBernoulli
/RelaxedOneHotCategorical
andBernoulli
/OneHotCategorical
depending on if the method is relaxation based or not.Also, the same Gumbel-Softmax trick holds for the distribution over the set of permutations - Plackett-Luce distribution. You can see successful applications in https://arxiv.org/abs/1911.10036, https://arxiv.org/abs/1903.08850.
Pitch
Tensorflow has tfp.distributions.PlackettLuce. I think it will be very cool to have it in the pytorch.
Additional context
For the efficient implementation we will need numerically stable implementation of LogCumSumExp function which is almost finished in PR https://github.com/pytorch/pytorch/pull/36308