Implement Log Energy

[zenna]

It would be cool to be able to include prior log densities

• It's required for #19
• Would possibly allow for better solution for jacobian transforms in SSMH and other inference procedures
• Would allow likelihood based inference

One concern is that reuse of a random variables would arbitrarily change the values. That is, if some random variable X is reused in many contexts, we only want to compute its density once (is this true?). With soft conditions, its possible to recompute the a constrained random variable many times, which changes the overall value, but in the end it doesn't really matter. But a likelihood is not the same as a soft constraint, it cannot be changed arbitrarily.

x = uniform(1:3)
n = normal(0, 1)
f_(w) = [normal(w) for i = 1:x(w)]
f = ciid(f_)

[zenna]

Another case that concerns me is that there could be some random variable that we execute in the trace and therefore it affects the log likelihood but it doesn't affect the outcome

x = normal(0, 1)
y = normal(0, 1)
function f_(w)
  x(w)
  y(w)
end

f = ciid(f_)

This example doesn't really pose a problem because x and y are uncorrelated. Therefore even though executing x changes the likelihood of the trace, it doesn't matter since it changes it uniformly over all values of y. If x and y were correlated, then it should affect that value. It seems like they not be a problem at all. It's akin to throwing in a bunch of sample statements in a WebPPL program. Harmless. Need to verify this is true.

[zenna]

Memoization means that duplicate executions are avoided but what happens with interventions (currently memoization is disabled when intervening).

Suppose i do

function x_(om)
  a = normal(om, 0, 1)
  b = sin(a) + normal(om, 3, 4)
end
x = ciid(x_)
y = replace(x, sin => cos)
z = x + y
rand((x, y, z))

[zenna]

Let's start with a simple example

x = 0.3julia
ϵ_ = ϵ ~ Normal(0, 1)
y = M .* x .+ C .+ ϵ_
y_ = 21.0
y_ = m * x + c + randn()
e3 = (y .== y_) # evidence
ypost = M |ᶜ e3

There's a few ways that I have been thinking about it

Propagation

Given that I know that e3 is true, I can invert the equality and get a value for y, i.e. y_

General idea was to define propagations of the form "Conditional on knowing some value $a$ propagate to some other values"

So maybe I could define a bunch of propagators

"Given that we have a value of `x`
propagate(X::typeof(e3), x_, w) = x ? y => y_(w)