Add docu in particular for VB derivation

[joergfunger]

I added Annikas derivation of the VB to a sphinx docu and copied also the intro from model calibration (because I think it fits much better here). The final section is not fully correct, but we are still discussing this (some of the derivations are not clear to me yet, so we should only merge it when the final section with the free energy is correctly done). But maybe this helps for the discussion and can be finally used to document our code. In order to build the documentation, make sure sphinx and spinx-rtd-scheme are installed (I changed the requirements file). Then run doit in the main folder. The updated docu should then be build in ./build/index.html. In the future, we could add this build to the runner and have the docu automatically build and linked in the Readme.

[TTitscher]

When built locally on my Ubuntu 20.04 LTS, one equation was not compiled correctly and the colors in the equations only affected one variable. That seems to be sphinx-specific. I fixed that by replacing the (IMO) correct usage of {\color{red} red text} with the (IMO) incorrect \color{red}{red text}, but it works now.

Apart from that, I noticed that the derived update equation reads:

\Lambda m = \Lambda0 m0 + sc Jk^T(k - Jk m)

If you just take the Chapell-equation (20)

\Lambda m = \Lambda0 m0 + sc J^T(k + J m)

and replace Jk = -J, you end up with

\Lambda m = \Lambda0 m0 - sc Jk^T(k - Jk m)  // or
\Lambda m = \Lambda0 m0 + sc Jk^T(-k + Jk m)

This is also what I implemented here and the test run correctly.

I'll continue trying to figure that out... Hints are welcome!

[aradermacher]

@TTitscher : You are right and I found the minus mistake. :)

[joergfunger]

Where do we update changes, the docu, the slides or the tech report?

[TTitscher]

Generally, whatever is in this repo, should be correct. Right now, this is only docs/variational_bayes_chappel.rst (right?) so that file should be updated.

[aradermacher]

I'm working on the update Sorry I'm not that fast

[aradermacher]

I tried to proof your free energy equation by derive it with respect to the parameters s,c,m,L which should result to the update equation. For s it works for Lamda I have a minus issue (again ...). The others are to difficult for me. If someone knows some tools to do it easily, please!

[joergfunger]

I checked the derivations, I have the impression that the second term in the eq. dF/dLambda has a wrong sign (+0.5*d(L^-1L)dL instead of -. I think this would give the update eq.

[aradermacher]

I also did the derivation with respect to m which yield to the other update equation. I will add it next week.

[TTitscher]

Sooo, would it be correct to summarize the changes as follows?

• The update equations were previously correct for J = -dk/dtheta. And the swap to J = dk/dtheta in #29 is also correctly implemented.
• The free energy formula from the paper is not correct and needs to be adapted.

[joergfunger]

Added building the docu using Readthedocs and is now integrated into the Readme. As for the test, it is difficult, since the F that we are computing is omitting the constant terms, thus it is not the model evidence (computed by prob taralli) or the complete F in eq 3 of Chappels paper.