Open dswah opened 8 years ago
on some real problems, we see that the negative log likelihood (with l1 penalty) increases. the increase happens as a result of the update to theta.
here is a plot of the difference in loss after the update to theta. it looks like sometimes the loss increases!
but the lambda updates are protected by the backtracking:
the loss should NEVER increase due to a Theta update because the objective is convex and we have a closed-form solution for the update.
on some real problems, we see that the negative log likelihood (with l1 penalty) increases. the increase happens as a result of the update to theta.
here is a plot of the difference in loss after the update to theta. it looks like sometimes the loss increases!
but the lambda updates are protected by the backtracking: