Effect of cross validation parameter on model metrics

[bkowshik]

In cross-validation, the cv parameter determines the cross-validation splitting strategy. Ex: If cv=3, it is a 3-fold cross-validation. I was curious to see the impact of the value of cv on the model metrics.

Workflow

• Load up a trained model
• Vary cv from 1 to 320 and run cross validation
• On each run, record the model metrics in precision, recall and f1 score

The following is the graph I got.

Questions

• What should be a good value of cv to use to consistently measure model performance?

---

cc: @anandthakker @batpad @geohacker

[bkowshik]

I am still unclear, 💭 on why this happened. Let's revisit this again when required.