XGBoost learning rate other than 1.0 gives bad results

h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Apache License 2.0

6.92k stars 2k forks source link

As reported on h2ostream: https://groups.google.com/forum/#!topic/h2ostream/F-E7Lzil284

There seems to be an issue on R for xgboost implementation:

Documentation for learn_rate and eta state that they are the same parameters, however, it is stated that they are given different default value (0 for eta and 0.1 for learn_rate) When using one of this parameter with other values than 1, algorithm doesn't seem to do anything (loss is very high even for 0.9 as learn_rate, compared to 1). Running on:

R.version _ platform x86_64-apple-darwin13.4.0 arch x86_64 os darwin13.4.0 system x86_64, darwin13.4.0 status major 3 minor 3.2 year 2016 month 10 day 31 svn rev 71607 language R version.string R version 3.3.2 (2016-10-31)

MSE on cross_validation with all others parameters fixed:

learn_rate_0_9 MSE = 10194

learn_rate_1 MSE = 75.8

PS: running on latest H2O stable version (3.5.10.2)

h2oai / h2o-3

XGBoost learning rate other than 1.0 gives bad results #11517