benchmark: Allow re-running jobs

mlr-org / mlr3

mlr3: Machine Learning in R - next generation

https://mlr3.mlr-org.com

GNU Lesser General Public License v3.0

927 stars 86 forks source link

benchmark: Allow re-running jobs #890

Closed pfistfl closed 1 week ago

pfistfl commented 1 year ago

Currently, if jobs in a benchmark() call fail, e.g. due to communication loss when parallelizing on a cluster it's unclear how to restart single lost jobs. I think we should either

be able to restart some runs (e.g. a row in the benchmark design)
tell people to use the batchtools backend for such cases

But in any way, how this is done should be documented a) in the benchmark docs and b) the appropriate section in the book.

be-marc commented 1 week ago

Added a reference for mlr3batchmark to benchmark() function