Closed SupplyChef closed 1 year ago
Hi @SupplyChef,
Thank you for trying out the example on GPs.
This error is because RxInfer.jl
tries to compute the free energy for you; there is no rule for free energy computation when handling missing data.
The quick fix would be to turn off free energy computations, e.g.
result = inference(
model = locallevel(n, 1, 1),
data = (y = y_data,),
free_energy = false
)
I hope @HoangMHNguyen can help you further with this issue.
I see. The reason I was using free energy is because I wanted to optimize the hyper-parameter of the model (σ², σ²_noise). I am thinking now of optimizing the hyper parameters without the missing data, and then inferring the missing data with fixed hyper-parameters. Do you think that's the best/reasonable way forward?
@SupplyChef Without going to much into the details - there is a workaround:
@marginalrule NormalMeanPrecision(:out_μ) (m_out::Missing, m_μ::NormalDistributionsFamily, q_τ::PointMass, ) = begin
out = @call_rule NormalMeanPrecision(:out, Marginalisation) (m_μ = m_μ, q_τ = q_τ)
return @call_marginalrule NormalMeanPrecision(:out_μ) (m_out = out, m_μ = m_μ, q_τ = q_τ)
end
Distributions.entropy(::Missing) = ReactiveMP.CountingReal(Float64, -1)
@average_energy NormalMeanPrecision (q_out::Missing, q_μ::Any, q_τ::Any) = begin
# Assume that a node with a missing point should not contribute to the Free Energy, you may change it though
return 0
end
I tried you model in this case and the Free Energy is being computed properly.
Details: This model runs with free_energy = false
, because the inference procedure does not require those marginals, but the Bethe Free Energy procedure does require. RxInfer cannot compute the joint marginal distribution given a missing data point, there is simply not enough information to compute those marginals. But it would be mathematically correct to factorize out missing points from the variational distribution. RxInfer does that where it can automatically (e.g. for datavar
s and constvar
s), but in your case it happens inside the graph. Generally speaking, support for missing data points is not fully implemented yet and one must be very careful as a lot of functionality is missing
😅. In my example I attempted to enforce mean-field approximation manually.
P.S.
Just in case, you can remove these rules if you don't use +
operator in you model:
@rule typeof(+)(:in1, Marginalisation) (m_out::Missing, m_in2::Any) = missing
@rule typeof(+)(:in1, Marginalisation) (m_out::Any, m_in2::Missing) = missing
Thank you for the work around and the explanation!
Can we close this one?
I think we can close this one because Dmitry has explained clearly everything.
Should be a part of the Sharp Bits section. But yea, we can close @albertpod
I am trying to modify the GP regression example to create a local level (forecasting) model. I tried to follow the Missing Data example to handle the missing data. However I get a warning about MarginalRuleMethodError and cannot find documentation/examples on how to proceed. I would echo issue #15 about having more examples for forecasting cases. Thank you!
My model is defined as:
The data is the same as in the GP regression example with some missing data for the prediction part:
I also added the following rules for missing data:
And call the inference:
When running, I am getting the following warning which I do not know how to handle. Do you have a suggestion? Thank you