Stacked Ensemble not winning in AutoML

h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Apache License 2.0

6.85k stars 1.99k forks source link

Here are some examples where the Stacked Ensemble is not winning the leaderboard in AutoML. We should investigate what’s happening here and see if we can improve the results by adjusting the metalearner.

The regression example [here |https://bradleyboehmke.github.io/HOML/stacking.html]with 80 models.
A binary classification example [here|https://0xdata.atlassian.net/browse/PUBDEV-7071] but the AUC is very high (0.999) so maybe this is not that big of a concern since that’s common.

More examples are summarized in this private notebook: [https://github.com/h2oai/h2oautoml-internal/blob/master/benchmarks/automlbenchmark/analyze_leaderboards.ipynb|https://github.com/h2oai/h2oautoml-internal/blob/master/benchmarks/automlbenchmark/analyze_leaderboards.ipynb|smart-link]

h2oai / h2o-3

Stacked Ensemble not winning in AutoML #8612