Implement higher precision gradients for Gamma, Beta, Dirichlet

probtorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

http://pytorch.org

Other

3 stars 1 forks source link

Implement higher precision gradients for Gamma, Beta, Dirichlet #49

Closed fritzo closed 6 years ago

fritzo commented 6 years ago

PRs https://github.com/pytorch/pytorch/pull/4369 https://github.com/pytorch/pytorch/pull/4421

@martinjankowiak has implemented higher-precision gradients for Gamma, Beta, and Dirichlet distributions based on Rice saddle point expansions. Derivations are here: Gamma, Beta and Dirichlet.

fritzo commented 6 years ago

@gchanan @colesbury When would be the best time for me to modify PyTorch's standard_gamma_grad() and dirichlet_grad()? If you're in the process of refactoring these then I can hold off for a couple weeks, otherwise I'll try to create a PR this week. (@martinjankowiak and I were testing these last week when Gamma.rsample() broke, hence my eagerness to get it fixed.)

gchanan commented 6 years ago

@fritzo all the changes I was planning to make in the short term are in, so go ahead!