search

stanfordmlgroup / ngboost

Natural Gradient Boosting for Probabilistic Prediction
Apache License 2.0
1.64k stars 214 forks source link

Using NGBoost for GAMs #158

Closed timothyb0912 closed 4 years ago

timothyb0912 commented 4 years ago

Hi there, thanks so much for putting this package together!

I want to use NGBoost to implement a generalized additive model. For example, I want to implement a model where the natural parameters of a given conditional distribution is given by f(x_1) + g(x_2) + h(x_1, x_2), where x_1 and x_2 are sets of features.

My desired approach would be to fit boosted tree ensembles for each of the natural parameters using just the features in x_1. Then I'd continue the procedure of fitting residuals of the objective function using just the features in x_2. Finally, I'd continue fitting residuals using both x_1 and x_2.

Could anyone give a brief demonstration of how this could be done or point out the tools / objects / methods that I should be using?

avati commented 4 years ago

Hi @timothyb0912 , you probably want to have a look straight at the core of the package - ngboost.py to see how you could do something like backfitting in the boosting framework. I imagine you could try to hide/mask features in each boosting stage in a particular pattern and also achieve similar algorithm as you describe?

alejandroschuler commented 4 years ago

Interesting idea @timothyb0912 ... I like @avati's suggestion of alternatively masking features as you go along. Would probably be the easiest thing to hack together given what we've got.