search

ReactiveBayes / ExponentialFamily.jl

ExponentialFamily.jl is a Julia package that extends the functionality of Distributions.jl by providing a collection of exponential family distributions and customized implementations.
https://reactivebayes.github.io/ExponentialFamily.jl/
MIT License
12 stars 2 forks source link

add fisher information for continuous Bernoulli #105

Closed ismailsenoz closed 1 year ago

ismailsenoz commented 1 year ago

This PR adds fisher information for continuous Bernoulli distribution.

Nimrais commented 1 year ago

Are you certain that the Fisher information for the vague continuous Bernoulli distribution is 0? I would expect it to be strictly positive simply because I expect Fisher information for CB is continuous (at least if we will fill the hole in 0.5).

κ = 0.500000001
dist = ContinuousBernoulli(κ)
ef = convert(KnownExponentialFamilyDistribution, dist)
η = getnaturalparameters(ef)

f_logpartition = (η) -> logpartition(KnownExponentialFamilyDistribution(ContinuousBernoulli, η))
autograd_information = (η) -> ForwardDiff.hessian(f_logpartition, η)
@info fisherinformation(ef), first(autograd_information(η))

(0.0, 15.749999998786821)

This behaviour is a bit suspicious. But maybe the second derivative is badly defined here.