openml-labs / gama

An automated machine learning tool aimed to facilitate AutoML research.
https://openml-labs.github.io/gama/master/
Apache License 2.0
93 stars 30 forks source link

Runtime warning #21

Closed joaquinvanschoren closed 5 years ago

joaquinvanschoren commented 5 years ago

I'm getting a lot of runtime warnings on the minimal example:

X, y = load_digits(return_X_y=True)
X_train, X_test, y_train, y_test= train_test_split(X, y, stratify=y, random_state=42)
automl = GamaClassifier(max_total_time=300, n_jobs=4)
automl.fit(X_train, y_train)
predictions = automl.predict(X_test)
print('accuracy', accuracy_score(y_test, predictions))
anaconda3/lib/python3.6/site-packages/sklearn/utils/deprecation.py:58: DeprecationWarning: Class Imputer is deprecated; Imputer was deprecated in version 0.20 and will be removed in 0.22. Import impute.SimpleImputer from sklearn instead.
  warnings.warn(msg, category=DeprecationWarning)
Could not create a new individual from 50 iterations of mate_new
anaconda3/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
Could not create a new individual from 50 iterations of mate_new
Could not create a new individual from 50 iterations of mate_new
Could not create a new individual from 50 iterations of mate_new

The DeprecationWarning can be fixed easily, I assume? The second I'm not so sure about.

'Could not create a new individual' is not very descriptive. Does this mean that there is something wrong with the algorithm? Is there something the user should do (e.g. use different hyperparameters)?

PGijsbers commented 5 years ago

Thanks for reporting these issues.

joaquinvanschoren commented 5 years ago

OK, cool.

Maybe also change the message to something like 'Notice: Gave up on mating X and Y. Could not create a new individual that was not evaluated before after 50 iterations'?

PGijsbers commented 5 years ago

I'm in favor of that 👍 the reason it is currently non-descriptive is that it is a shared error message (the mate_new can also be generate_new or mutate_new), but that should be fixed.

PGijsbers commented 5 years ago

Closing this issue because the DeprecationWarning is fixed (by upgrading minimum version of scikit-learn and using the new SimpleImputer) and the error message has a separate issue.