Nested list changes behaviour of `reduceResultsDataTable`?

[giuseppec]

Consider the following code

library(OpenML)
library(batchtools)

unlink("test", recursive = TRUE)
reg = makeExperimentRegistry(file.dir = "test", packages = c("mlr", "OpenML"), seed = 123)

tid = 59
addProblem(name = paste0("oml", tid), data = list(tid = tid), seed = tid)

addAlgorithm("run", fun = function(job, instance, data, lrn) {
  otask = getOMLTask(data$tid)
  r = runTaskMlr(otask, lrn)
  list(run = r)
})

ades = data.table(
  lrn = list(
    makeLearner("classif.featureless"),
    makeLearner("classif.rpart")
  )
)

addExperiments(algo.designs = list(run = ades))

submitJobs()

done = findDone()
res = reduceResultsDataTable(ids = done, fun = function(job, res) {
  z = as.list(getBMRAggrPerformances(res$run$bmr, as.df = TRUE))
  z$tid = job$problem$data$tid
  return(z)
})

res

The github version returns

   job.id result
1:      1 <list>
2:      2 <list>

The CRAN version returns

   job.id task.id          learner.id acc.test.join timetrain.test.sum timepredict.test.sum tid
1:      1    iris classif.featureless     0.3333333               0.00                 0.00  59
2:      2    iris       classif.rpart     0.9400000               0.05                 0.01  59

Will the reduceResultsDataTable behviour change in the next batchtools version so that my current code won't be backward compatible?

[mllg]

1. The changes are somewhat necessary. Previous versions automatically un-flatten such lists, depending on the result of a flaky heuristic. This often leads to confusion or errors in the analysis
2. In the long run, I currently see no way to remain backward compatible. I could first change the behavior to signal deprecation warnings though, if this helps. Do I break something critical?

[mllg]

NB: try flatten(res).

[mllg]

This is now on CRAN. Users get a notice on attach.