search

reiinakano / scikit-plot

An intuitive library to add plotting functionality to scikit-learn objects.
MIT License
2.43k stars 284 forks source link

Custom Scorer for CV inside plot_learning_curve #26

Closed david1309 closed 7 years ago

david1309 commented 7 years ago

Hello,

I am using cross-validation with a particular metric, Kappa score, rather than the standard accuracy metric.

cross_val_score(clf, x_train, y_train, scoring=kappa_scorer, cv=kf, n_jobs=-1)

I would like to to set the CV done inside the plot_learning_curve method for each set of train_sizes to use the Kappa Scorer and not the accuracy score. I would also like to use the Kappa Scorer to evaluate the models performance for the training set. Is there any way to set this in the plot_learning_curve method ?

david1309 commented 7 years ago

I guess inside plot_learning_curve you are using sklearn functions learning_curve. Maybe be if you allow passing an argument scorer in your plot_learning_curve functions, this might be further passed to sklearns learning_curve and in this way one can score both, training and validation data with a custom scorer.

reiinakano commented 7 years ago

Hi @david1309 ! You're right, I can just expose that as an additional parameter. I'll be working on it but feel free to submit a PR if you want.

reiinakano commented 7 years ago

@david1309, long overdue, but closed by #55