dl with ae: get ava.lang.UnsupportedOperationException: Trying to predict with an unstable model.

[exalate-issue-sync[bot]]

parse attached file and run -

buildModel 'deeplearning', {"model_id":"deeplearning-ba49a0c7-cf9f-4c01-944c-8953015cb013","training_frame":"Key_Frame__covkts.hex","ignored_columns":[],"ignore_const_cols":true,"response_column":"V55","activation":"Rectifier","hidden":[200,200],"epochs":10,"variable_importances":false,"balance_classes":false,"max_confusion_matrix_size":20,"max_hit_ratio_k":10,"checkpoint":"","train_samples_per_iteration":-2,"adaptive_rate":true,"input_dropout_ratio":0,"l1":0,"l2":0,"loss":"Automatic","score_interval":5,"score_training_samples":10000,"score_validation_samples":0,"score_duty_cycle":0.1,"autoencoder":true,"class_sampling_factors":[],"max_after_balance_size":5,"overwrite_with_best_model":true,"target_ratio_comm_to_comp":0.02,"seed":-2551274355908727000,"rho":0.99,"epsilon":1e-8,"max_w2":"Infinity","initial_weight_distribution":"UniformAdaptive","classification_stop":0,"regression_stop":0.000001,"score_validation_sampling":"Uniform","diagnostics":true,"fast_mode":true,"force_load_balance":true,"single_node_mode":false,"shuffle_training_data":false,"missing_values_handling":"MeanImputation","quiet_mode":false,"sparse":false,"col_major":false,"average_activation":0,"sparsity_beta":0,"max_categorical_features":2147483647,"reproducible":false,"export_weights_and_biases":false}

get - Got exception 'class java.lang.RuntimeException', with msg 'java.lang.RuntimeException: java.lang.RuntimeException: water.DException$DistributedException: from /127.0.0.1:54321; by class hex.deeplearning.DeepLearningModel$2; class java.lang.UnsupportedOperationException: Trying to predict with an unstable model.' java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: water.DException$DistributedException: from /127.0.0.1:54321; by class hex.deeplearning.DeepLearningModel$2; class java.lang.UnsupportedOperationException: Trying to predict with an unstable model. at hex.deeplearning.DeepLearning$DeepLearningDriver.trainModel(DeepLearning.java:382) at hex.deeplearning.DeepLearning$DeepLearningDriver.buildModel(DeepLearning.java:232) at hex.deeplearning.DeepLearning$DeepLearningDriver.compute2(DeepLearning.java:146) at water.H2O$H2OCountedCompleter.compute(H2O.java:698) at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104) Caused by: java.lang.RuntimeException: java.lang.RuntimeException: water.DException$DistributedException: from /127.0.0.1:54321; by class hex.deeplearning.DeepLearningModel$2; class java.lang.UnsupportedOperationException: Trying to predict with an unstable model. at hex.deeplearning.DeepLearningModel.doScoring(DeepLearningModel.java:710) at hex.deeplearning.DeepLearning$DeepLearningDriver.trainModel(DeepLearning.java:352) ... 8 more Caused by: java.lang.RuntimeException: water.DException$DistributedException: from /127.0.0.1:54321; by class hex.deeplearning.DeepLearningModel$2; class java.lang.UnsupportedOperationException: Trying to predict with an unstable model. at water.MRTask.getResult(MRTask.java:489) at water.MRTask.doAll(MRTask.java:386) at water.MRTask.doAll(MRTask.java:382) at hex.deeplearning.DeepLearningModel.scoreAutoEncoder(DeepLearningModel.java:816) at hex.deeplearning.DeepLearningModel.doScoring(DeepLearningModel.java:542) ... 9 more Caused by: water.DException$DistributedException: from /127.0.0.1:54321; by class hex.deeplearning.DeepLearningModel$2; class java.lang.UnsupportedOperationException: Trying to predict with an unstable model. at hex.deeplearning.DeepLearningModel.score_autoencoder(DeepLearningModel.java:923) at hex.deeplearning.DeepLearningModel.access$200(DeepLearningModel.java:30) at hex.deeplearning.DeepLearningModel$2.map(DeepLearningModel.java:824) at water.MRTask.compute2(MRTask.java:641) at water.MRTask.compute2(MRTask.java:599) at water.MRTask.compute2(MRTask.java:599) at water.MRTask.compute2(MRTask.java:599) ... 6 more

[exalate-issue-sync[bot]]

Arno Candel commented: Rectifier can lead to unstable model, try using Tanh, or add l1 or set max_w2 to a value around 10.

[DinukaH2O]

JIRA Issue Migration Info

Jira Issue: PUBDEV-1529 Assignee: New H2O Bugs Reporter: Nidhi Mehta State: Resolved Fix Version: N/A Attachments: Available (Count: 1) Development PRs: N/A

Attachments From Jira

Attachment Name: covkts.csv Attached By: Nidhi Mehta File Link:https://h2o-3-jira-github-migration.s3.amazonaws.com/PUBDEV-1529/covkts.csv