A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
I'm tuning a LightGBM model using its goss version. However, I'm running into an issue where I cannot set the top_rate and other_rate in a way that their sum is greater than or equal to 1.
From the first paragraph in page 6 of the NIPS paper (emphasis in bold are mine),
Specifically, GOSS firstly sorts the data instances according to the absolute value of their
gradients and selects the top a×100% instances. Then it randomly samples b×100% instances from the rest of the data
So my understanding is that there should be no restriction in the values of these parameters, besides both being within [0,1].
For example. a data with 1,000 observations and top_rate=0.6 and other_rate=0.5 would select the first 600 observations (sorted by the absolute value of their gradients) and then would pick 200(=0.5*0.4*600) observations from the rest of the data.
However, based on the error, it seems that the other_rate is not picking what's left, but rather picking the share of original sample.
Any ideas?
Thanks!
Environment info
Operating System: UNIX
CPU/GPU model: AWS EC2 r5.24xlarge
C++/Python/R version: Python v3.6
LightGBM version or commit hash: 2.3.0
Error message
LightGBMError: Check failed: config_->top_rate + config_->other_rate <= 1.0f at /tmp/pip-req-build-zww_zxel/compile/src/boosting/goss.hpp, line 63 .
Reproducible examples
Below is one hyperparameter configuration that leads to the above error
Hello,
I'm tuning a LightGBM model using its goss version. However, I'm running into an issue where I cannot set the
top_rate
andother_rate
in a way that their sum is greater than or equal to 1.From the first paragraph in page 6 of the NIPS paper (emphasis in bold are mine),
So my understanding is that there should be no restriction in the values of these parameters, besides both being within [0,1].
For example. a data with 1,000 observations and
top_rate=0.6
andother_rate=0.5
would select the first 600 observations (sorted by the absolute value of their gradients) and then would pick 200(=0.5*0.4*600) observations from the rest of the data.However, based on the error, it seems that the other_rate is not picking what's left, but rather picking the share of original sample.
Any ideas?
Thanks!
Environment info
Operating System: UNIX
CPU/GPU model: AWS EC2 r5.24xlarge
C++/Python/R version: Python v3.6
LightGBM version or commit hash: 2.3.0
Error message
Reproducible examples
Below is one hyperparameter configuration that leads to the above error