Final steps / thoughts

[mohlerm]

So we're coming close to the exam :)

Is there anything you think that should be added or modified? As discussed with @passuf, I think it would make sense to try to finish the exam summary until 16:00 (UTC+2). @kilianrisse @Robin-des-Bois

[Robin-des-Bois]

Some possibilities:

• Jensen's inequality.
• Maybe some small section about general EM
• Dictionary learning: how to perform step 2

What do you think?

[mohlerm]

• Jensen's inequality can be seen as a tool for the proofs and derivations in this lecture. Since they are not part of the summary I don't think we need Jensen as well. Don't get me wrong, I think knowing its details is very important for the exam, I just think that one has to understand it anyway to use it properly. However, it is on the exam already (kind of), in section 5, MLE. I think this is enough and paired with general understanding of Jensen one should be able to use it.
• EM...maybe, not sure what to put there though...
• Yes, I think this is the same as #10

[Robin-des-Bois]

For EM I would like to have something along the lines of

\subsection*{Latent variable}
We denote the latent variable indicating the component the point is sampled from by Z, which takes on values in $\{1,...,k\}$.

\subsection*{E-step: Posterior probabilities}
$\gamma_j^t(x_i) = P(Z=j|x_i, \theta_t) = \frac{P(x_i|Z=j, \theta_t) P(Z=j|\theta_t)}{Z}$

\subsection*{M-step: maximizing expected log likelihood}
$\mathbb{E}_{\gamma^t}[\log P(\mathcal{D;\theta})] = 
\mathbb{E}_{\gamma^t}[\log \Pi_{i=1}^nP(x_i,z_i;\theta)] = $ \\
$\sum_{i=1}^n \mathbb{E}_{\gamma^t}[\log P(x_i,z_i;\theta)] = $ \\
$\sum_{i=1}^n \sum_{j=1}^k \gamma_j^t(x_i) \log (P(x_i|z_i=j;\theta) P(z_i=j;\theta))$ \\
$\theta_{t+1} = \underset{\theta}{\operatorname{argmax}} \mathbb{E}_{\gamma^t}[\log P(\mathcal{D;\theta})]$

But it would need quite some space. :(

[kilianrisse]

I don't really think this is gonna fit on the summary and for me personally this is not of big use; I think the other stuff on the summary is more important.

[mohlerm]

I think most of these things are mentioned implicitely already and yeah, there isn't really space for it.