search

davidrosenberg / mlcourse

Machine learning course materials.
https://davidrosenberg.github.io/ml2018
570 stars 267 forks source link

Alternate Proof of Square Sum #35

Open brett1479 opened 7 years ago

brett1479 commented 7 years ago

This isn't an issue, but just another idea. Suppose you have x1+x2+...+xn = c and you want to solve this while minimizing ||x||_2. Suppose xi > xj. Then -ei + ej is a descent direction that lies within the constraint hyperplane.

davidrosenberg commented 7 years ago

This is for showing that l2 spreads weight evenly? and showing it's a descent direction is a cute inner product with the gradient. I like it - where do we put it?

brett1479 commented 7 years ago

Well, I thought of it while watching your Elastic Net slides (you give a picture proof for 2 variables). Not sure if it belongs there in class, or maybe as a concept check/hw exercise.

davidrosenberg commented 7 years ago

This sounds good -- want to write this up? We can use it for concept check or move it into homework #2.

brett1479 commented 7 years ago

So my argument goes like this. If not all equal, then not minimum. But there must be a minimum (compactness). Thus proven. Is the compactness part beyond what we should discuss?