entropy

[cscherrer]

Having an efficient and robust way to compute the entropy of a model is important in general, and can accelerate approaches like variational inference. With this PR we can do, e..g,

julia> p = @model r,s begin
           p_bad ~ Beta(1,3) |> iid(s)
           bad ~ For(s) do j
                   Bernoulli(p_bad[j], check_args=false)
               end

           fpr ~ Beta(1,2) |> iid(r)
           tpr ~ Beta(2,1) |> iid(r)
           pos_rate = hcat(fpr, tpr)
           y ~ For(r,s) do i,j
                   Bernoulli(pos_rate[i, bad[j] + 1], check_args=false)
               end
       end;

julia> entropy(p(r=5,s=14))
33.46395343775838

Addition of entropy also required some new particle-related functionality.

This PR also changes the prior transformation so the variables passed are interpreted as pointing to observed data, and adds a corresponding likelihood. So, e.g.,

julia> m = @model n begin
           α ~ Gamma()
           β ~ Gamma()
           θ ~ Beta(α,β)
           x ~ Binomial(n, θ)
       end;

julia> prior(m, :x)
@model begin
        β ~ Gamma()
        α ~ Gamma()
        θ ~ Beta(α, β)
    end

julia> likelihood(m, :x)
@model (n, θ) begin
        x ~ Binomial(n, θ)
    end

[codecov-io]

Codecov Report

Merging #218 into master will decrease coverage by 1.22%. The diff coverage is 3.50%.

@@            Coverage Diff             @@
##           master     #218      +/-   ##
==========================================
- Coverage   31.58%   30.36%   -1.23%     
==========================================
  Files          43       44       +1     
  Lines        1317     1370      +53     
==========================================
  Hits          416      416              
- Misses        901      954      +53     

Impacted Files	Coverage Δ
src/Soss.jl	100.00% <ø> (ø)
src/core/dependencies.jl	100.00% <ø> (ø)
src/core/utils.jl	40.86% <0.00%> (-0.90%)	:arrow_down:
src/particles.jl	14.54% <0.00%> (-5.46%)	:arrow_down:
src/primitives/entropy.jl	0.00% <0.00%> (ø)
src/transforms/markovblanket.jl	2.12% <0.00%> (ø)
src/transforms/basictransforms.jl	33.33% <11.11%> (-66.67%)	:arrow_down:
src/distributions/for.jl	74.10% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3df3e0b...3de4b2f. Read the comment docs.