search

ClimbsRocks / auto_ml

[UNMAINTAINED] Automated machine learning for analytics & production
http://auto-ml.readthedocs.io
MIT License
1.64k stars 310 forks source link

Custom eval metrics #325

Closed duoan closed 6 years ago

duoan commented 7 years ago

Hi, guy: I want to know is there is a api that I can set my own eval metrics function.

ClimbsRocks commented 7 years ago

there is. it's in undocumented alpha release, so i'd love your feedback on it!

from sklearn.metrics import precision_score
ml_predictor.train(data, scoring=precision_score)

caveats:

  1. this is a scoring metric, not an objective function.
  2. undocumented alpha release- with all the usual caveats that go with that.
  3. we use it in a few places to decide between trained models, so make sure it conforms to the sklearn api (y_true, y_pred), as well as higher values being better scores (so many evaluation metrics need to be multiplied by -1, like RMSE)

would love your feedback on it!

duoan commented 6 years ago

Good.