search

microsoft / LightGBM

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
https://lightgbm.readthedocs.io/en/latest/
MIT License
16.51k stars 3.82k forks source link

[python-package] Do we have the support for merging of multiple lightgbm models? #5339

Open jinyyy666 opened 2 years ago

jinyyy666 commented 2 years ago

I have an application that trains a boosting sequence with tree models and linear models like: linear-model, trees, linear-model, and trees. In the end, I want to combine all the tree models into one.

I do not find the function to do that in lightgbm.

In catboost library, they have a function called: sum_models: https://catboost.ai/en/docs/concepts/python-reference_sum_models

which does this.

jinyyy666 commented 2 years ago

One way I can think of is to just write a wrapper and mock all the functions currently existing for the lightgbm.Booster class.

shiyu1994 commented 2 years ago

@jinyyy666 Thanks for using LightGBM. Unfortunately, we don't have an official approach to merge multiple LightGBM models.

If the goal is just to combine the prediction results of multiple boosting model, I think it can be achieved by simply adding the prediction results from different models together.

However, if the goal is the blending different base learners (trees or linear models) during ONE boosting process, I think it can hardly be achieved by existing GBDT tools without modifying the source code.

If you have any further problems, please feel free to post them here. Thanks!