search

h2oai / h2o-tutorials

Tutorials and training material for the H2O Machine Learning Platform
http://h2o.ai
1.48k stars 1.01k forks source link

GridSearch for Ordinal Regression fails #64

Open raviraju opened 6 years ago

raviraju commented 6 years ago

Used dataset (http://h2o-public-test-data.s3.amazonaws.com/bigdata/laptop/glm_ordinal_logit/ordinal_multinomial_training_set.csv) from tutorial to configure grid search

hyper_parameters = {'alpha': [0.01,0.3,0.5], 'lambda': [1e-5,1e-6,1e-7,1e-8]} gs = H2OGridSearch(H2OGeneralizedLinearEstimator(family='ordinal'), grid_id='ordinal_grid', hyper_params=hyper_parameters) gs.train(x=list(range(0,10)), y="C11", training_frame=train) gs.show()

glm Grid Build progress: |████████████████████████████████████100%

NotImplementedError Traceback (most recent call last)

in () 4 grid_id='ordinal_grid', 5 hyper_params=hyper_parameters) ----> 6 gs.train(x=list(range(0,10)), y="C11", training_frame=train) 7 gs.show() /apps/anaconda3/lib/python3.6/site-packages/h2o/grid/grid_search.py in train(self, x, y, training_frame, offset_column, fold_column, weights_column, validation_frame, **params) 206 x = list(xset) 207 parms["x"] = x --> 208 self.build_model(parms) 209 210 /apps/anaconda3/lib/python3.6/site-packages/h2o/grid/grid_search.py in build_model(self, algo_params) 221 if is_auto_encoder and y is not None: raise ValueError("y should not be specified for autoencoder.") 222 if not is_unsupervised and y is None: raise ValueError("Missing response") --> 223 self._model_build(x, y, training_frame, validation_frame, algo_params) 224 225 /apps/anaconda3/lib/python3.6/site-packages/h2o/grid/grid_search.py in _model_build(self, x, y, tframe, vframe, kwargs) 277 first_model_json = h2o.api("GET /%d/Models/%s" % 278 (rest_ver or 3, grid_json['model_ids'][0]['name']))['models'][0] --> 279 self._resolve_grid(grid.dest_key, grid_json, first_model_json) 280 else: 281 if len(failure_messages_stacks)>0: /apps/anaconda3/lib/python3.6/site-packages/h2o/grid/grid_search.py in _resolve_grid(self, grid_id, grid_json, first_model_json) 286 287 def _resolve_grid(self, grid_id, grid_json, first_model_json): --> 288 model_class = H2OGridSearch._metrics_class(first_model_json) 289 m = model_class() 290 m._id = grid_id /apps/anaconda3/lib/python3.6/site-packages/h2o/grid/grid_search.py in _metrics_class(model_json) 756 model_class = H2ODimReductionGridSearch 757 else: --> 758 raise NotImplementedError(model_type) 759 return model_class 760 NotImplementedError: Ordinal
ABartzGit commented 6 years ago

Hi, Ravi. You received the error because "family" is not a gridable parameter. Please see: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/grid-search.html#glm-hyperparameters

Let me know if this helps and/or if you have any other questions. -Angela

raviraju commented 6 years ago

Hi Angela,

I understand "family" isn't a gridable parameter, however I have used it to instantiate GLM model for Ordinal Regression and have specified hyper_parameters dictionary for grid parameters.

I used example in https://github.com/h2oai/h2o-3/blob/master/h2o-docs/src/product/tutorials/random%20hyperparmeter%20search%20and%20roadmap.md#examples-Python for reference

wendycwong commented 6 years ago

Hi Ravi:

I fixed the problem for Python and added tests to make sure R also works. Here is the PR https://github.com/h2oai/h2o-3/pull/2429. Will push it into master as soon as it passed the test.

Thank you for catching it, Wendy

wendycwong commented 6 years ago

Merged into master.

raviraju commented 6 years ago

Hi Wendy,

Thanks for fix, Can u help me pull these changes using conda

ABartzGit commented 6 years ago

Hi Ravi, Thank you again for pointing out the bug. The fix is in the master branch and will be available on anaconda.org with the 3.20.0.1 release. You can test this in regular Python by downloading the nightly build: http://h2o-release.s3.amazonaws.com/h2o/master/4305/index.html.

Sincerely, Angela