cdebacco
commented
2 years ago Thanks for the questions, I try to be synthetic with the answers.
Main
- Mean Field: this can have different specific meaning in different contexts. For instance, in the Curie Weiss model mean field had the meaning of an interaction observed by a given variable, which was the average one over all the neighbors. In practice, it was as if we were neglecting the fluctuations around the mean, as in Eq. 11-12 L4. From the point of view of VI, we can simply state that MF is assuming a fully-factorized distribution over each individual random variable.
- BP is an algorithmic implementation, while Bethe is more robust theoretically. You can use either of them, but one may be more convenient to implement than the other. See L6 and L7 for extensive discussion about both.
- For the exam: all the State rules that apply for entering the University apply also to the day of the exam. Hence please be ready to show 3G certification and personal ID before the exam starting time. FFP2 mask should be carried during the whole duration of the exam.
- Sorry for the confusion, an example using VEM was originally planned but given the lack of time I decided to skip it. So please discard the sentence "We will cover this in the next lectures".
- Yes, N2 should be in "Auf der Morgenstelle 16", Hörsäle Erdgeschoss (this is the address I have)
Rest
- Z unknown: I simply meant that these variables are not observed (they are latent), so we can only estimate their posterior probability, or get a point estimate of them (e.g. using MLE).
- De Finetti's can also be seen in L1 eq 21. The main point of that equation is that the right-hand size, the argument inside the integral, is all nicely factorized, given some latent variable. In LDA we saw an application of this, where the topic proportions theta is the parameter allowing this factorization into conditional probabilities. The fact that order of words does not matter (as in the bag of word assumption), allows to apply that theorem. In practice, one works with the joint P(w,z,theta), hence with the argument of the integral (thus avoiding calculating the integral), which has that nice factorized structure.
- No need to know the notation of exponential families. In case, I would write down the formula directly on the exam sheet.
- No need for a cheat sheet, as everything you need to know will be already presented in the exam sheet. For instance, no need to remember the shape of a Dirichelet or Poisson distribution, in case this will be written down in the sheet.
- The exam can include topics from both Lectures and Tutorials. From the Tutorial I can only sample questions that do not require coding for the exam. There are several questions that were solved in the tutorials that were extending the Lecture to some more specific scenarios or applications, that I did not have time to cover during the lecture. So all of these should be taken in consideration.
- For preparing at the exam I would make sure to understand the main ideas of the most important concepts learned in these class. I do not expect you to know small details, I am more interested to test if you understood the main ideas. There may be one or two derivation exercises, but these are usually short, hence if you reviewed the lectures and practice with the tutorials you should be able to make the derivation even if you see that exercise for the first time. The Exam template uploaded in this repository is a good example of prototypical questions.
meier-johannes94
Most important questions
I still don‘t have a real understanding of what mean field is. Or in other words I don‘t have a proper intuition yet for it. I (mostly) get the two approaches of lecture 4. But where is the similarity that makes you speak of both of them as „mean field“? How to explain mean field simply (like to a child) without being inacurate? What would you expect as an answer if you would ask in the exam: „What is mean-field?“
We have been combining belief propagation and Bethe in lecture 6. I have to admit I don‘t understand yet, where Bethe‘s value-add is to Belief propagation. I get that Bethe has sound theory and Belief propagation is fast, but where comes Bethe into play s.t. things are possible that have not been possible bgefore?
Is there anything special to keep in mind for the exams from an organization point of view (like 3G, ...)?
In lecture 8 you wrote: "Another approach is to use variational methods like Variational Expectation Maximization (VEM). We will cover this in the next Lectures". Where can I find that (lecture number, chapter if possible :-))?
The exam is written in N2. This is Morgenstelle, correct?
Rest
Lecture 2: You are saying on page 4 that the set of Z is unknown. Why is that the case? I mean we defined the generative process and so we also defined what values for Z are possible, so we know the set, right? Or is it that for Z we might have an infinite number of options s.t. we would need an integral and that integral is again to difficult to solve? Or is it maybe that for most of the Zs sum_z {p(x, z|theta} is small anyway, so it is not relevant. So we need to find Zs for which that probability is high?
I don‘t get De Finett‘s theorem on LDA yet (lecture 12), no matter how often I read that 3 lines. :-D, it would be great if you could explain that again (maybe elaborately if needed). I also don‘t get the relation between the formula (9) and the two sentences above fully.
What should we know about exponential families for the exam?
We are not allowed to bring a cheat sheet or anything like that, correct?
I have the impressions that the exam questions are „very“ mostly related to the lectures and less to the exercises. What parts of the exercises should we look the closest at?
Do you have any additional recommendations on where to focus when preparing for the exam?
Thanks a lot of answering!