k-fold cross validation

[manmeet3591]

Is there a way to perform k-fold cross validation with this setup?

[Yura52]

If you don't need hyperparameter tuning, then you can store k possible splits as k separate datasets, and then average the metrics for all of them.

However, if you need hyperparameter tuning, then you have two options:

• tune hyperparameters using only one of the k splits
• for each set of hyperparameters, run the training and evaluation on all the splits and treat the average metric as the score. This will require modifications to the code, namely, you will need to add explicit support for passing many datasets to one script.

Does this help?

[manmeet3591]

Thank you. Yes, this definitely helps.