Open jaheba opened 4 years ago
What would be the advantage?
Generally, I think evaluation code should be strictly separate from model (estimator/predictor) code, because they have different functions and should therefore be separated.
Generally, I think evaluation code should be strictly separate from model (estimator/predictor) code
I agree, but I guess the idea is to still keep the code separate by having the evaluate
method in the base Predictor
class.
@jaheba one observation: the way you put it
predictor.evaluate(test_dataset)
would be a replacement for backtest_metrics
rather than make_evaluation_predictions
.
I like this proposal because:
Estimator
class which will do the training and call the evaluation method of the resulting predictor.backtest_metrics
/make_evaluation_predictions
duality by means of optional arguments to select whether to compute only predictions or also evaluation metrics.@jaheba should we punt this to the 0.6 milestone?
Instead of using
make_evaluation_predictions
predictors should know how to evaluate themselves: