search

h2oai / xgboost

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Flink and DataFlow
Other
44 stars 26 forks source link

h2o.xgboost doesn't support more than 920+ variables while modelling. #40

Closed ghost closed 6 years ago

ghost commented 6 years ago

I am getting below error if my predictors are exceeding 920 in h2o.xgboost() version 3.18.05. It was working perfectly fine till 3.17.x version of h2o. Please find error below. Any suggestion or help will be appreciated.

DistributedException from localhost/127.0.0.1:54321: 'null', caused by java.lang.NullPointerException at water.MRTask.getResult(MRTask.java:478) at water.MRTask.getResult(MRTask.java:486) at water.MRTask.doAll(MRTask.java:390) at water.MRTask.doAll(MRTask.java:386) at ml.dmlc.xgboost4j.java.XGBoostScoreTask.runScoreTask(XGBoostScoreTask.java:45) at hex.tree.xgboost.XGBoostModel.makePreds(XGBoostModel.java:367) at hex.tree.xgboost.XGBoostModel.makeMetrics(XGBoostModel.java:343) at hex.tree.xgboost.XGBoostModel.makeMetrics(XGBoostModel.java:337) at hex.tree.xgboost.XGBoostModel.doScoring(XGBoostModel.java:387) at hex.tree.xgboost.XGBoost$XGBoostDriver.doScoring(XGBoost.java:454) at hex.tree.xgboost.XGBoost$XGBoostDriver.scoreAndBuildTrees(XGBoost.java:357) at hex.tree.xgboost.XGBoost$XGBoostDriver.buildModelImpl(XGBoost.java:330) at hex.tree.xgboost.XGBoost$XGBoostDriver.buildModel(XGBoost.java:260) at hex.tree.xgboost.XGBoost$XGBoostDriver.computeImpl(XGBoost.java:250) at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:206) at water.H2O$H2OCountedCompleter.compute(H2O.java:1263) at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: java.lang.NullPointerException at hex.tree.xgboost.XGBoostUtils.dense(XGBoostUtils.java:313) at hex.tree.xgboost.XGBoostUtils.convertChunksToDMatrix(XGBoostUtils.java:281) at ml.dmlc.xgboost4j.java.XGBoostScoreTask.map(XGBoostScoreTask.java:137) at water.MRTask.compute2(MRTask.java:657) at water.MRTask.compute2(MRTask.java:591) at water.H2O$H2OCountedCompleter.compute1(H2O.java:1266) at ml.dmlc.xgboost4j.java.XGBoostScoreTask$Icer.compute1(XGBoostScoreTask$Icer.java) at water.H2O$H2OCountedCompleter.compute(H2O.java:1262) ... 5 more

Error: DistributedException from localhost/127.0.0.1:54321: 'null', caused by java.lang.NullPointerException

Pscheidl commented 6 years ago

Hello @rushabhpatel . Your issue should be fixed in latest H2O nightly release.

Please download it here: http://s3.amazonaws.com/h2o-release/h2o/master/4315/index.html and give it a try. If not. please file an issue in H2O-3 project, because your issue is related to H2O-3: https://0xdata.atlassian.net/projects/PUBDEV/summary

Best regards, Pavel