It’s really nice to include cross-validation in the evaluation approach by default.
Question Is it true cross-validation (train on 80%, test on remaining 20% for 5-fold) or a pseudo-cross- validation (train on 100%, but rotate which 20% is the evaluation)? It seems like having an option for true cross-validation might be nice, because then you can use that on both incremental and batch algorithms.
Comment Clarify that it's pseudo cross-validation; true is already implemented in dibs and AG but would have to be enabled; would have to be implemented for scratch for TP, puddle - it does not make sense to implement for baselines
It’s really nice to include cross-validation in the evaluation approach by default.
Question Is it true cross-validation (train on 80%, test on remaining 20% for 5-fold) or a pseudo-cross- validation (train on 100%, but rotate which 20% is the evaluation)? It seems like having an option for true cross-validation might be nice, because then you can use that on both incremental and batch algorithms.
Comment Clarify that it's pseudo cross-validation; true is already implemented in dibs and AG but would have to be enabled; would have to be implemented for scratch for TP, puddle - it does not make sense to implement for baselines