Closed desilinguist closed 1 year ago
Patch coverage: 100.00
% and project coverage change: +0.05
:tada:
Comparison is base (
143ff09
) 95.19% compared to head (10469c3
) 95.24%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
Last year, scikit-learn added functionality to include model fit times when computing learning curves since – in addition to the model's performance – it's also quite useful to know how the long the model takes to train as more training data was added. This PR now adds the same functionality to SKLL.
skll.utils.train_and_score()
function now measures the model fit time for every model trained as part of a learning curve experiment.learning_curve
experiment. The first is the usual "score curve" that shows the training and cross-validation scores as more training data is added. The newly-added second plot is a "time curve" that shows how the model fit times change as more training data is added. The format for this new curve's name is:<experiment>_<featureset>_times.png
.skll.experiments.output.generate_learning_curve_plots
function. It now only pre-processes the score and time data to create data frames. The two curves (score and time) are now generated by two private functions:skll.experiments.output._generate_learning_curve_score_plots
andskll.experiments.output._generate_learning_curve_time_plots
.As always, the best way to review is to try this out in the examples. As a starting point, if you want to replicate the same example, you can modify the Titanic example's
learning_curve.cfg
file as shown below and then look at theTitanic_Learning_Curve_all.png
andTitanic_Learning_Curve_all_times.png
files in theoutput
directory.This PR closes #556.