search

automl / auto-sklearn

Automated Machine Learning with scikit-learn
https://automl.github.io/auto-sklearn
BSD 3-Clause "New" or "Revised" License
7.54k stars 1.28k forks source link

Is it possible to integrate a metric of imblearn as a scorer? #786

Open moelliDo opened 4 years ago

moelliDo commented 4 years ago

I would like to use the geometric mean as a metric to optimize classification models trained with the fit() method. I thought that the autosklearn.metrics.make_scorer() would allow to define the geometric mean as a scorer like: classifier.fit(X_train, y_train, feat_type=feat_type, metric=autosklearn.metrics.make_scorer("gm", imblearn.metrics.geometric_mean_score)) as the imblearn-package is fully compatible with sklearn.

However, after the model has been fitted, the sprint statistics indicate, that the definition as I did it does not seem to be working:

auto-sklearn results: Dataset name: 6b31930a65e59cca700a5844fbab91a0 Metric: gm Best validation score: 0.000000 Number of target algorithm runs: 187 Number of successful target algorithm runs: 82 Number of crashed target algorithm runs: 74 Number of target algorithms that exceeded the time limit: 18 Number of target algorithms that exceeded the memory limit: 13

Is it somehow possible to define the geometric mean as a metric to optimize the model?

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs for the next 7 days. Thank you for your contributions.