shane-kercheval
commented
6 years ago - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5001642/
- Due to the absence of a completely separate test set it is not possible to claim that a particular model, i.e. a particular set of weights and classifier parameters, could in the future be used to classify unseen data [2]. In addition, the chosen parameters and models may vary between cross-validation iterations, making it impossible to select one set of parameters or one model as the final choice. In other words, a separate model and a separate set of parameters are chosen in each iteration and choosing any one of them would mean returning to a simple cross-validation and testing approach which would annihilate the advantage gained by nested cross-validation.
- https://sebastianraschka.com/faq/docs/evaluate-a-model.html
- inner folds are used to optimize the hyperparameters of each model (e.g., using gridsearch in combination with k-fold cross-validation. If your model is stable, these m models should all have the