search

JuliaAI / MLJLinearModels.jl

Generalized Linear Regressions Models (penalized regressions, robust regressions, ...)
MIT License
81 stars 13 forks source link

generalized / group lasso #153

Open adienes opened 1 year ago

adienes commented 1 year ago

there are a few other regularized forms of Lasso / ElasticNet I would find useful, like group lasso, trend filtering, generalized lasso (& generalized Tikhonov). is this compatible with existing design of solvers?

tlienart commented 1 year ago

Hmm so if I'm not mistaken some of what you describe comes down to a penalty that looks like $\sum_i P_i(\Gamma_i\theta) $ where $\theta$ is the vector of coefficients, $P_i$ are basic penalisations that are mostly already there and $\Gamma_i$ is some matrix e.g. that masks a set of the vector or whatever (e.g. TV would be another one).

I think that for that kind of penalty, the extension should be fairly straightforward with a lot of re-use of existing code.

For even more generic stuff (e.g. if someone was interested in actually solving an Lp penalty with 0<=p<1) this will typically require ad-hoc solvers and while it might be in scope, it would expand the code surface to maintain quite a bit. I'd say let's discuss case by case?