mllg / batchtools

Tools for computation on batch systems
https://mllg.github.io/batchtools/
GNU Lesser General Public License v3.0
172 stars 51 forks source link

removeExperiments removes to much? #113

Closed jakob-r closed 7 years ago

jakob-r commented 7 years ago
library(batchtools)
set.seed(030)

unlink("~/nobackup/test", recursive = TRUE)
reg = makeExperimentRegistry(file.dir = "~/nobackup/test")

probleme = list("a", "b")

pdes = lapply(probleme, function(p) {
  addProblem(name = p, data = p, fun = function(...) list(...), seed = reg$seed)
  res = data.frame(fold = 1:10)
})
names(pdes) = probleme

algo.rep1 = function(job, data, instance, x) {
  rep(paste(data, x), instance$fold)
}

algo.rep2 = function(job, data, instance, x) {
  rep(paste(data, x), instance$fold)
}

addAlgorithm(name = "rep1", fun = algo.rep1)
addAlgorithm(name = "rep2", fun = algo.rep2)

ades = list(
  rep1 = expand.grid(x = LETTERS[1:10]),
  rep2 = expand.grid(x = letters[1:10])
)

addExperiments(pdes, ades)

submitJobs()

getStatus()
Syncing 400 files ...
Status for 400 jobs:
  Submitted : 400 (100.0%)
  Queued    :   0 (  0.0%)
  Started   : 400 (100.0%)
  Running   :   0 (  0.0%)
  Done      : 400 (100.0%)
  Error     :   0 (  0.0%)
  Expired   :   0 (  0.0%)
ids.rep1 = findExperiments(algo.name = "rep1")
ids.rep2 = findExperiments(algo.name = "rep2")
showLog(id = ids.rep1[1,])
removeExperiments(ids.rep2)
Removing 200 Experiments ...
Cleaning up 200 job definitions ...
Removing 200 obsolete result files ...
Removing 400 obsolete log files ...
Removing 0 obsolete job collection files ...
Removing 0 job description files ...
Removing 0 external directories of unsubmitted jobs ...
Removing 0 resource specifications ...
Removing 0 tags ...
showLog(id = ids.rep1[1,])
Error in readLog(id, reg = reg) : 
  Log file for job with id 1 not available
reduceResultsList(ids = ids.rep1)
Error in gzfile(file, "rb") : cannot open the connection
In addition: Warning message:
In gzfile(file, "rb") :
  cannot open compressed file '/home/richter/nobackup/test/results/1.rds', probable reason 'No such file or directory'
mllg commented 7 years ago

Fixed via #114.