Currently we directly raise any errors from tune or sklearn. We should add better error messages so the user knows how to resolve these errors. There are two types of errors that we need to better messages for:
Errors with Tune. For example
ray.tune.error.TuneError: Insufficient cluster resources to launch trial: trial requested 1 CPUs, 1 GPUs, but the cluster has only 16 CPUs, 0 GPUs, 5.47 GiB heap, 1.86 GiB objects (1.0 node:192.168.2.228). Pass `queue_trials=True` in ray.tune.run() or on the command line to queue trials until the cluster scales up or resources become available.
Users should not be exposed to Tune internals and a more user-friendly message should be provided.
Sklearn errors. For example
Traceback (most recent call last):
File "_test.py", line 14, in <module>
tune_search.fit(X, y)
File "/Users/rliaw/miniconda3/lib/python3.7/site-packages/sklearn/model_selection/_search.py", line 710, in fit
self._run_search(evaluate_candidates)
File "/Users/rliaw/miniconda3/lib/python3.7/site-packages/sklearn/model_selection/_search.py", line 1151, in _run_search
evaluate_candidates(ParameterGrid(self.param_grid))
File "/Users/rliaw/miniconda3/lib/python3.7/site-packages/sklearn/model_selection/_search.py", line 689, in evaluate_candidates
cv.split(X, y, groups)))
File "/Users/rliaw/miniconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 335, in split
for train, test in super().split(X, y, groups):
File "/Users/rliaw/miniconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 80, in split
for test_index in self._iter_test_masks(X, y, groups):
File "/Users/rliaw/miniconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 692, in _iter_test_masks
test_folds = self._make_test_folds(X, y)
File "/Users/rliaw/miniconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 663, in _make_test_folds
% (self.n_splits))
ValueError: n_splits=5 cannot be greater than the number of members in each class.
However, it's not clear to the user what the fix should be. In this case the user needs to reduce the cv parameter in their Tune searcher object.
For both of these, we should catch the errors on a case by case basis and output better messages.
Currently we directly raise any errors from tune or sklearn. We should add better error messages so the user knows how to resolve these errors. There are two types of errors that we need to better messages for:
Errors with Tune. For example
Users should not be exposed to Tune internals and a more user-friendly message should be provided.
Sklearn errors. For example
However, it's not clear to the user what the fix should be. In this case the user needs to reduce the
cvparameter in their Tune searcher object.For both of these, we should catch the errors on a case by case basis and output better messages.