Type

[zenna]

There are several issues that this issue is about.

At a high level we need to determine the structure of the Omega types. Let me just stat the different issues

Intervene

The first issue I noticed was a bug in intervene.

function test_intervene_diff_parents()
  x = 1 ~ Normal(0, 1)
  function yy(ω)
    x_ = x(ω)
    Normal(x_, 1)(ω)
  end
  y = 2 ~ yy
  x2 = 3 ~ x
  yi = y |ᵈ (x => ω -> 100.0)
  yi = y |ᵈ (x => ω -> 100.0)
  yi2 = y |ᵈ (x2 => ω -> 100.0)
  yi_, yi2_ = randsample(ω -> (yi(ω), yi2(ω)))
  function test(ω)
    @show y1 = yi(ω)
    @show y2 = yi2(ω)
    y1, y2
  end
  @test yi_ != yⁱ2_
end

The root problem is that we are using the same scope for two random variables with different parameters, Normal(0, 1) and Normal(100, 1). So due the way LazyOmega works, whichever is executed first will be the value for both of them, which is clearly wrong.

The fundamental question is what should be the semantics of distribution families.

The three answers I came up with

1. Space of iid distributions. I think this is not a good iea.
2. Like (old) Omega. Non parameterized primitives and random variable families are a class of functions on the same primtiives.
3. Tha most radical -- omega is values store (presumably integral) seeds, and then when we want to get the value of a random variable for different parametrs we execute the sampling process with the seed initialised in the same place but different parameters.

What are the additional requirements

1. For inference we need to control the random variable, this could be the inmput or output. Preferably we'd be able to do both.

2. We'd like it to be compatible (at least some of the time) with methods that use gradients.

3. We'd like it to be compatible with as many random variable types as possible.

Concerning the first point, the main issue i encountered was consistency. Suppose we have

mu = 10
X = Normal(mu 1)
X’ = X | do(mu => 20)

Which, for inference, do we control the output of? If we control the value of X the value of X' is determined. If omega just contains the seed then theres no way to control the output of a variable