Open joegaotao opened 4 years ago
@joegaotao I'm not sure how to do that.
Note to myself: http://www.montefiore.ulg.ac.be/~ernst/uploads/news/id63/extremely-randomized-trees.pdf
Just as a note, lightgbm added this feature quite recently. I think it would certainly be of value.
Yup. I recently unified the evaluation procedure, partly for categorical data support, partly for features like this.
Just a small bump in case this might be something that could make it into 1.4?
Depends. Right now the categorical data support is my priority. I will see how much time left after sorting out categorical data.
The variety of mdels plays an important role in model ensemble. I tried some parameters such as "colsample_bytree, colsample_bynode" to make model more stable and different, but trees still grow by some criterion, resulting in the similar models. However, I tried the combination "extratree+lgb", and randomized tree could be used as the certain feature embeding tool that improves model variety. So I suggest adding extremely randomized tree as base learner.