search

davidrosenberg / mlcourse

Machine learning course materials.
https://davidrosenberg.github.io/ml2018
560 stars 264 forks source link

Clean up "equivalence" of penalty and constraint form regularization #78

Open davidrosenberg opened 5 years ago

davidrosenberg commented 5 years ago

There's something going on that needs clarification. Suppose w^ is an unconstrained minimizer of phi(w), and suppose Omega(w^)=R. Say phi(w) is a quadratic objective for a linear system that has a null space. Then there's another minimizer w^ that has Omega(w^)=r>R. So w^** is a solution to a constrained optimization problem, but not of the penalized form. What assumption is not met in the equivalence theorem from the homework?

Even if w^** is not attained as a solution to a penalty form problem, say something about the corresponding linear function being attained?

brett1479 commented 5 years ago

w^** is a solution of the penalized form with lambda=0.