search

APMLA-2021 / APMLA-WS_21-22_material

MIT License
32 stars 5 forks source link

Lagrange multiplier in L2 #11

Closed SebastianBruijns closed 2 years ago

SebastianBruijns commented 2 years ago

Hi, I don't really understand the calculations involving the Langrage multiplier, in particular I'm unsure how to get from eq. (8) to eq. (9). I could at most see how lambda = - N_k / pi_k, but the text says lambda = -N I'm also not sure what is meant with "take the sum over k to enforce the constraint". Thank you very much for any help.

cdebacco commented 2 years ago

I imagine you are referring to L2. If you multiply both sides of Eq8 by pi_k, and then sum over k (to enforce the constraint), you will get that the numerator inside the sum over n of the first term equals the denominator (they are both sums_k(pi_k x Gauss(x_n|mu_k,E_k)). Hence that ratio gives 1, and you simply remain with sum_n 1 = N. This leads to lambda = -N. I hope this clarifies.

SebastianBruijns commented 2 years ago

I see, what confused me was that I thought it really just meant multiplying by pi_k, which I thought was OK since we're taking the derivative with respect to pi_k, so it is a specific pi_k. But it seems more accurate to say we're multiplying with sum_k (pi_k), which is obviously one. Thank you.