search

felix-clark / ndarray-glm

Rust library for linear, logistic, and generalized linear model regression
MIT License
22 stars 0 forks source link

Proper LASSO regression #32

Closed felix-clark closed 1 year ago

felix-clark commented 1 year ago

Right now L1 regularization is implemented with a smoothed V-curve potential. It would be better to use a LASSO approach, which should reduce over-stepping and actually encourage parameters to stay near zero instead of bouncing back and forth.

felix-clark commented 1 year ago

https://stats.stackexchange.com/questions/3511/how-to-apply-the-iteratively-reweighted-least-squares-irls-method-to-the-lasso

felix-clark commented 1 year ago

I'm pursuing an approach using ADMM to split off the non-differentiable part of the Lagrangian. This necessitates tracking additional hidden variables with each IRLS iteration, so some refactoring is necessary to make this compatible.