"chaining"

[TTitscher]

Out of curiosity, I quickly implemented a test case here that generates n_data (currently 2) datasets (linear model m * xs + c) and compares

• a single VB run on a combined model error with all n_data:

  param:         MVN with 
                ├── mean: [ 6.757e+00  1.002e+01]
                ├── std:  [ 6.782e-01  4.187e-02]
  noise:         {'noise0': Gamma with 
                └── mean: (69.11101249683999, 9)}

• n_data VB runs with one of the data sets and taking the posterior as prior for the next run (thus "chaining" :smile: ):

  param:         MVN with 
                ├── mean: [ 6.672e+00  1.003e+01]
                ├── std:  [ 7.352e-01  4.543e-02]
  noise:         {'noise0': Gamma with 
                └── mean: (67.5849803323776, 9)}

Potential issue: The results do not match. Should they?

[joergfunger]

I would have assumed this to give the same result, since the theoretical solution is exactly the same, since the model is linear (thus the linearization does not depend on m).

[TTitscher]

Even if you decrease the tolerance such that it is never reached and VB iterates 50 times -- and I would expect 50 fixed-point iterations to be enough -- you get the same result as above...

[joergfunger]

That could be checked with the analytical example, we could take 2 data points and compute the exact solution with a given prior, and then do the same thing with the sequential approach. Would Isabela or Abbas being interested to check the standard deviation part in the example for the analytical solution Thomas implemented and verify, if this is true for the subsequent iteration, and then, why this is not the case for our implementation. Maybe we still have a bug in our implementation that leads to wrong results, and then any discussion of results with a larger complexity might be obsolete.

[TTitscher]

If I skip the noise update (I know that this actually requires a new derivation of the equations...), the parameter posteriors match. So I also expect investigating the standard deviation part in the example for the analytical solution to be fruitful.