start working on validation support

[sebffischer]

• [ ] The GraphLearner / Graph should null the test roles unless there is a PipeOp that makes use of them. Otherwise we might end up making predictions that are not needed. Maybe we want to achieve this differently / change this in mlr3?

PipeOps to look at:

• [ ] PipeOpChunk
• [ ] PipeOpFeatureUnion
• [ ] PipeOpImpute
• [ ] PipeOpOVRSplit
• [x] PipeOpTuneThreshold (not really anything to do)
• [x] PipeOpLearner (not really anything to do)
• [ ] PipeOpLearnerCV (not a lot to do)
• [ ] PipeOpTrafo
• [ ] PipeOpTaskPreproc: train and predict stage now need to work a little differently even for stuff like PipeOpPreprocSimple

[sebffischer]

This does not work because of the same reason bootstrapping does not work with mlr3pipelines. Consider the case where a task has overlapping "use" and "test" rows, e.g. task$row_roles$use of c(1, 2) and task$row_roles$test and this task should be preprocessed by PipeOpTaskPreproc.

We want to treat the row 2 during train and 2 during predict differently, as we want to apply the training preprocessing to the former and the prediction preprocessing to the latter. This does not work, because the tasks's data backend has only 3 rows and we want to cbind a backend that has 4 rows.

[mb706]

https://github.com/mlr-org/mlr3pipelines/issues/706