search

pyro-ppl / numpyro

Probabilistic programming with NumPy powered by JAX for autograd and JIT compilation to GPU/TPU/CPU.
https://num.pyro.ai
Apache License 2.0
2.09k stars 227 forks source link

Add entropy implementations. #1787

Closed tillahoffmann closed 2 months ago

tillahoffmann commented 2 months ago

This PR adds entropy implementations, using scipy.stats distributions as the reference implementation for tests where possible. There is some overlap with #1706.

Currently, only one entropy implementation is available (for the LowRankMultivariateNormal). It might be useful to switch entropy to use lazy_property as for mean and variance. That would make the interface consistent (entropy is a property of the distribution like mean or variance), and it would allow caching if the evaluation is more involved. What do you think, @fehiepsi?

tillahoffmann commented 2 months ago

That makes sense regarding consistency with pytorch/pyro. I've updated the betaln normalization constant.