search

biaslab / ForneyLab.jl

Julia package for automatically generating Bayesian inference algorithms through message passing on Forney-style factor graphs.
MIT License
149 stars 35 forks source link

Natural parameter distributions #189

Closed ThijsvdLaar closed 2 years ago

ThijsvdLaar commented 2 years ago

Implement natural parameter definitions from https://github.com/semihakbayrak/ForneyLab.jl/tree/CVI Additionaly:

codecov-commenter commented 2 years ago

Codecov Report

Merging #189 (eb89500) into master (208861e) will increase coverage by 0.22%. The diff coverage is 98.59%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #189      +/-   ##
==========================================
+ Coverage   86.50%   86.72%   +0.22%     
==========================================
  Files          95       95              
  Lines        4395     4461      +66     
==========================================
+ Hits         3802     3869      +67     
+ Misses        593      592       -1
Impacted Files Coverage Δ
src/factor_nodes/log_normal.jl 87.80% <80.00%> (+3.59%) :arrow_up:
...c/engines/julia/update_rules/nonlinear_extended.jl 100.00% <100.00%> (ø)
.../engines/julia/update_rules/nonlinear_unscented.jl 89.52% <100.00%> (ø)
src/factor_nodes/bernoulli.jl 84.21% <100.00%> (+1.85%) :arrow_up:
src/factor_nodes/beta.jl 87.50% <100.00%> (+1.38%) :arrow_up:
src/factor_nodes/categorical.jl 77.19% <100.00%> (+1.72%) :arrow_up:
src/factor_nodes/dirichlet.jl 87.50% <100.00%> (+1.25%) :arrow_up:
src/factor_nodes/gamma.jl 85.00% <100.00%> (+1.66%) :arrow_up:
src/factor_nodes/gaussian.jl 100.00% <100.00%> (ø)
src/factor_nodes/poisson.jl 93.33% <100.00%> (+1.33%) :arrow_up:
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 208861e...eb89500. Read the comment docs.

albertpod commented 2 years ago

Looks good. But why do we add this to FL?

EDIT: for PR#188

ThijsvdLaar commented 2 years ago

Great, thanks; it's some prep work for natural gradient descent. I tried to refactor Semih's work (#183) in smaller chunks that fit my head.

semihakbayrak commented 2 years ago

Thanks @ThijsvdLaar for your contribution! I liked how you implemented StandardDist function directly with distribution and variate types rather than their instances. For the logpdf function of exponential family (EF) of distributions, how about implementing just one function: log(h(x)) + transpose(ϕ(x))*η - logNormalizer(dist,η) in probability_distributions.jl file instead of defining them separately for the distributions? We can then implement components of logpdf separately for each distribution, as it has been done for logNormalizer, so as to use them in other applications when necessary. For example, I can imagine that sufficient statistics is useful in many applications and would be nice to have this function in FL.

ThijsvdLaar commented 2 years ago

I agree that implementing a single log-pdf function is more beautiful from a mathematical point of view, but from a coding point-of view it would add complexity. We would have to define h and \phi functions for each distribution type, which are not used anywhere else outside of the log-pdf function. Since there is currently no case for reuse of these statistics, I would prefer the more direct approach.