search

h2oai / h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
http://h2o.ai
Apache License 2.0
6.94k stars 2k forks source link

AutoML Process fails on AWS EMR Machines with python 2.7.12 but same code works fine locally #11860

Open exalate-issue-sync[bot] opened 1 year ago

exalate-issue-sync[bot] commented 1 year ago

AutoML Process fails on AWS EMR Machines - python 2.7.12 - (however the exact same code works fine on my machine with Python 2.7.13)

Import a sample binary outcome train/test set into H2O

... train = h2o.import_file(“https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv“) Parse progress: |█████████████████████████████████████████████████████████████████████████████| 100% test = h2o.import_file(“https://s3.amazonaws.com/erin-data/higgs/higgs_test_5k.csv”) Parse progress: |█████████████████████████████████████████████████████████████████████████████| 100%

Identify predictors and response

... x = train.columns y = “response” x.remove(y)

For binary classification, response should be a factor

... train[y] = train[y].asfactor() test[y] = test[y].asfactor()

Run AutoML for 30 seconds

... aml = H2OAutoML(max_runtime_secs = 30) aml.train(x = x, y = y, ... training_frame = train, ... leaderboard_frame = test) AutoML progress: |████████████████████████████████████████████████████████████████████████████| 100% Leaderboard for project_name: automl_py_1_sid_950f: models sorted in order of auc, best first

model_id auc logloss

0 StackedEnsemble_0_AutoML_20171005_214734 0.764594 0.576976 1 XRT_0_AutoML_20171005_214734 0.745879 0.593952 2 DRF_0_AutoML_20171005_214734 0.745831 0.593528 H2OTypeError: Argument python_obj should be a None | list | tuple | dict | numpy.ndarray | pandas.DataFrame | scipy.sparse.issparse, got H2OTwoDimTable Error in sys.excepthook: Traceback (most recent call last): File “/usr/local/lib/python2.7/site-packages/h2o/utils/debugging.py”, line 95, in _except_hook _handle_soft_error(exc_type, exc_value, exc_tb) File “/usr/local/lib/python2.7/site-packages/h2o/utils/debugging.py”, line 225, in _handle_soft_error args_str = _get_args_str(func, highlight=highlight) File “/usr/local/lib/python2.7/site-packages/h2o/utils/debugging.py”, line 316, in _get_args_str s = str(inspect.signature(func))[1:-1] AttributeError: ‘module’ object has no attribute ‘signature’

Original exception was: Traceback (most recent call last): File ““, line 3, in File “/usr/local/lib/python2.7/site-packages/h2o/automl/autoh2o.py”, line 269, in train self._fetch() File “/usr/local/lib/python2.7/site-packages/h2o/automl/autoh2o.py”, line 307, in _fetch self._leaderboard = h2o.H2OFrame(res[“leaderboard_table”].as_data_frame())[1:] File “/usr/local/lib/python2.7/site-packages/h2o/frame.py”, line 90, in init assert_is_type(python_obj, None, list, tuple, dict, numpy_ndarray, pandas_dataframe, scipy_sparse) File “/usr/local/lib/python2.7/site-packages/h2o/utils/typechecks.py”, line 457, in assert_is_type skip_frames=skip_frames) h2o.exceptions.H2OTypeErrorLeaderboard for project_name: automl_py_1_sid_950f: models sorted in order of auc, best first

model_id auc logloss

0 StackedEnsemble_0_AutoML_20171005_214734 0.764594 0.576976 1 XRT_0_AutoML_20171005_214734 0.745879 0.593952 2 DRF_0_AutoML_20171005_214734 0.745831 0.593528 : Argument python_obj should be a None | list | tuple | dict | numpy.ndarray | pandas.DataFrame | scipy.sparse.issparse, got H2OTwoDimTable

exalate-issue-sync[bot] commented 1 year ago

Michal Malohlava commented: Pandas are missing in the Python environment.

exalate-issue-sync[bot] commented 1 year ago

Michal Malohlava commented: However, we need to fix code - it should not expect Pandas to be available

hasithjp commented 1 year ago

JIRA Issue Migration Info

Jira Issue: PUBDEV-4984 Assignee: New H2O Bugs Reporter: Avkash Chauhan State: Open Fix Version: N/A Attachments: N/A Development PRs: N/A