search

dswah / pyGAM

[HELP REQUESTED] Generalized Additive Models in Python
https://pygam.readthedocs.io
Apache License 2.0
876 stars 160 forks source link

GAMSEL implementation #191

Open owlas opened 6 years ago

owlas commented 6 years ago

I think an open source implementation of the GAMSEL model in python would be great. R is ahead of the curve here. I'm not sure if the model would belong in pyGAM since the current IRLS solver is likely (I haven't been over the details of the GAMSEL optimisation method yet) not sufficient for the model.

Is this something that the developers would be interested in including? It would be better to have a single reliable source for GAMs in python!

Links

dswah commented 6 years ago

@owlas agreed. it would be great to have some automatic feature selection.

i only just skimmed this paper, and it looks very cool. like you suggested, it seems that the objective function requires a more specialized optimization scheme (blockwise coordinate descent), and some additional complexity in general...

an alternative could be the one laid out in [1] to add an additional l2 / ridge penalty (with its own smoothing parameter) on each term, and then use the non-negative garrote to do the variable selection.

this seems to me like the most practical route to getting variable selection in pyGAM. However, as a prerequisite, pyGAM needs to be able to do automatic smoothing parameter selection.

[1] Marra, Wood, 2011. Practical variable selection for generalized additive models

owlas commented 6 years ago

Marra, Wood (2011) is going in at the top of my reading list.

danieliong commented 3 years ago

Is this still of interest and has anybody started working on this? I've looked through the repo and haven't seen anything so far but just asking to make sure. I would be interested in trying to implement something for variable selection.

I've skimmed the R implementation and it looks like the main parts of the algorithm are written in Rcpp. I don't know a whole lot about Rcpp but I'm thinking it might not be too difficult to rewrite it in Cython/C++.

nickcorona commented 3 years ago

Is this still of interest and has anybody started working on this? I've looked through the repo and haven't seen anything so far but just asking to make sure. I would be interested in trying to implement something for variable selection.

I've skimmed the R implementation and it looks like the main parts of the algorithm are written in Rcpp. I don't know a whole lot about Rcpp but I'm thinking it might not be too difficult to rewrite it in Cython/C++.

Please do. I'm in desperate need of this functionality, but don't have the know-how to implement it.