h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
http://h2o.ai
Apache License 2.0
6.86k stars 1.99k forks source link

checkpoint: when run with nfolds get AIOOB in gbm #14846

Open exalate-issue-sync[bot] opened 1 year ago

exalate-issue-sync[bot] commented 1 year ago

parse any data, run gbm with nfolds and call this model as a checkpoint in a later model get-

Got exception 'class java.lang.RuntimeException', with msg 'water.DException$DistributedException: from /192.168.1.11:54321; by class hex.tree.gbm.GBM$GBMDriver$GammaPass; class java.lang.ArrayIndexOutOfBoundsException: -1' java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.11:54321; by class hex.tree.gbm.GBM$GBMDriver$GammaPass; class java.lang.ArrayIndexOutOfBoundsException: -1 at water.MRTask.getResult(MRTask.java:489) at water.MRTask.doAll(MRTask.java:386) at water.MRTask.doAll(MRTask.java:382) at hex.tree.gbm.GBM$GBMDriver.buildNextKTrees(GBM.java:393) at hex.tree.gbm.GBM$GBMDriver.buildModel(GBM.java:179) at hex.tree.SharedTree$Driver.compute2(SharedTree.java:217) at water.H2O$H2OCountedCompleter.compute(H2O.java:1001) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: water.DException$DistributedException: from /192.168.1.11:54321; by class hex.tree.gbm.GBM$GBMDriver$GammaPass; class java.lang.ArrayIndexOutOfBoundsException: -1 at hex.tree.SharedTree.chk_weight(SharedTree.java:387) at hex.tree.gbm.GBM.access$6800(GBM.java:24) at hex.tree.gbm.GBM$GBMDriver$GammaPass.map(GBM.java:573) at water.MRTask.compute2(MRTask.java:641) ... 6 more

buildModel 'gbm', {"model_id":"gbm-ef7ce817-20f8-4c98-ba1f-14f02732afb3","training_frame":"Key_Frame__birds.hex","nfolds":"2","response_column":"S","ignored_columns":[],"ignore_const_cols":true,"ntrees":50,"max_depth":5,"min_rows":10,"nbins":20,"nbins_cats":1024,"seed":5919652325077650000,"learn_rate":0.1,"distribution":"gaussian","score_each_iteration":false,"fold_assignment":"AUTO","r2_stopping":0.999999,"build_tree_one_node":false,"checkpoint":"","keep_cross_validation_predictions":false,"nbins_top_level":1024}

buildModel 'gbm', {"model_id":"gbm","training_frame":"Key_Frame__birds.hex","nfolds":"2","response_column":"S","ignored_columns":[],"ignore_const_cols":true,"ntrees":"500","max_depth":5,"min_rows":10,"nbins":20,"nbins_cats":1024,"seed":5919652325077650000,"learn_rate":0.1,"distribution":"AUTO","score_each_iteration":false,"fold_assignment":"AUTO","r2_stopping":0.999999,"build_tree_one_node":false,"checkpoint":"gbm-ef7ce817-20f8-4c98-ba1f-14f02732afb3","keep_cross_validation_predictions":false,"nbins_top_level":1024}

DinukaH2O commented 1 year ago

JIRA Issue Migration Info

Jira Issue: PUBDEV-1885 Assignee: Michal Malohlava Reporter: Nidhi Mehta State: Open Fix Version: N/A Attachments: Available (Count: 1) Development PRs: N/A

Attachments From Jira

Attachment Name: birds.csv Attached By: Nidhi Mehta File Link:https://h2o-3-jira-github-migration.s3.amazonaws.com/PUBDEV-1885/birds.csv