Add another SP message passing example w ForneyLab in lesson 12 or 13

[bertdv]

At end of lesson 14, add the EM code example for GMM from lesson 9 again, but now learn the parameters by EM message passing. Print the message passing schedule and check that the result (after MP execution) is (about) the same as the result of lesson 9.

[marcocox]

EM message passing is not (yet) built into ForneyLab, so we would have to implement it specifically for this class. Let me know if you think this is useful to add before the class starts.

[bertdv]

No, EM for GMM is not needed. I changed the title. Last year I noticed that FFGs were hard to understand. So I dropped lesson 14 on EM as message passing altogether. I think we need more working examples of sum-product MP though, just to get the concept of MP across. Now we only have one (awesome) Kalman filter example. Perhaps sum-product MP for state estimation in HMMs would be useful. Or logistic regression. Let's think about 1 extra example and add it to either lesson 12 or 13. Not really necessary that it is ready before classes start (although that would be great). In the end, I ll probably end up making minor updates as the class progresses so we'll have to generate another PDF at the end of class.

[marcocox]

Sorry for the delayed response.

• HMM state estimation: possible, but the required "state transition" node is not yet available in ForneyLab. Idem for the required likelihood node. However, they are available in Anouk's repo, so it should be easy to port. Possible downside is that you will have to introduce these new nodes (and discrete distributions), which might not help to clarify the general concept of MP.

• Logistic regression requires EP. However, standard linear ridge regression is possible with pure sum-product MP, we only need the "dot-product" node (special case of the GainNode). So it's easy to add a linear regression example.

Which option has your preference? I'll implement it over the weekend.

[bertdv]

Thanks for looking into this. I think for now the linear ridge regression example will do. Eventually (not necessarily this year) we will want to add HMM as well. My experience last year was that we had too much theory and not enough examples. Perhaps better to introduce this topic through a few examples.

[bertdv]

... over the weekend https://www.youtube.com/watch?v=dsRhe2vyHlY

[marcocox]

@bertdv I added a first version of the linear regression code example in a new branch:

https://github.com/bertdv/AIP-5SSB0/blob/linear_regression_mp/lessons/Bayesian_linear_regression_as_message_passing/Bayesian%20linear%20regression%20as%20message%20passing.ipynb

I'm not sure how you want to wrap the code in the lessons/notebooks, so the text around the code is still quite scarce. Let me know if I should modify anything. I guess you can add it to an existing lesson, or create a new lesson for it.

[bertdv]

Thanks a lot. I had a quick look and it looks fine. Let me think about how to incorporate it.

[bertdv]

I ve added your piece to the end of the factor graph lesson. thanks! http://nbviewer.jupyter.org/github/bertdv/AIP-5SSB0/blob/master/lessons/12_factor_graphs/Factor-Graphs-and-Message-Passing-Algorithms.ipynb