project-codeflare / codeflare

Simplifying the definition and execution, scaling and deployment of pipelines on the cloud.
https://codeflare.dev
Apache License 2.0
222 stars 36 forks source link

Add support for scoring metrics to Runtime.cross_validate() #21

Open raghukiran1224 opened 3 years ago

raghukiran1224 commented 3 years ago

Overview

Current implementation does not accept user specified scoring metric(s). For example, cross_val_score(pipeline, X_test, y_test, scoring="neg_mean_squared_error", cv=10) The list of sklearn model evaluation metrics are listed here https://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter

Acceptance Criteria

One common use case documented below is supported by codeflare pipeline. https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfitting_overfitting.html#sphx-glr-auto-examples-model-selection-plot-underfitting-overfitting-py

Questions

Assumptions

Reference

raghukiran1224 commented 3 years ago

created by @yuanchi2807