search

dmlc / treelite

Universal model exchange and serialization format for decision tree forests
https://treelite.readthedocs.io/en/latest/
Apache License 2.0
730 stars 98 forks source link

Ability to combine tree models #577

Closed stephenpardy closed 2 months ago

stephenpardy commented 2 months ago

Based on the documentation (and my personal experience) it is not possible to combine multiple tree models in treelite without manually editing the tree parameters in ways that are prone to errors. I am wondering if there is any plan to support this feature or interest in this if I were to look into it more.

This would unlock some interesting boosting and ensembling techniques not currently supported by XGBoost or other major tree modeling packages by allowing users to train multiple models and then combine them together after the fact.

hcho3 commented 2 months ago

@stephenpardy Treelite already offers the method to concatenate tree models. Does this method does what you want?

# model1, model2, model3 are treelite.Model objects
concatenated_model = treelite.Model.concatenate([model1, model2, model3])
# returns the combined model

See the example at https://github.com/dmlc/treelite/blob/d8f12bc4dc9490f0cf632f38179f1faa70a773a2/tests/python/test_model_concat.py

stephenpardy commented 2 months ago

Yes! Apologies for missing that. I got confused by a comment in the docs:

Currently, it is not possible to add or remove trees using the field accessor API.

This looks like it works perfectly for what I want.