search

cerlymarco / linear-tree

A python library to build Model Trees with Linear Models at the leaves.
MIT License
338 stars 54 forks source link

learning_rate in boosting #17

Closed fipelle closed 2 years ago

fipelle commented 2 years ago

Hi, is there a way for setting the learning_rate in the boosting regressors and classifiers?

EDIT: Also, is LinearBoostingRegressor fitting a linear regression first and then boosting the residual via regression trees or boosting via a series of linear regression trees?

cerlymarco commented 2 years ago

Hi, thanks for your feedback.

No, here learning_rate is not involved.

Linear Boosting is a two-stage learning process. Firstly, a linear model is trained on the initial dataset to obtain predictions. Secondly, the residuals of the previous step are modeled with a decision tree using all the available features. The tree identifies the path leading to the highest error (i.e. the worst leaf). The leaf contributing to the error the most is used to generate a new binary feature to be used in the first stage.

In this sense, I think the code is quite explicative.

If you support the project don't forget to leave a star ;-)