Closed HenrikBengtsson closed 8 years ago
The following now works:
library("async")
> plan(batchjobs, backend="~/.BatchJobs.R", resources=list(nodes="1:ppn=2", vmem="1gb"))
> v %<=% { availableCores() }
> v
PBS
2
where ~/.BatchJobs.R
is:
cluster.functions <- makeClusterFunctionsTorque("~/.BatchJobs,torque.tmpl")
cluster.functions$listJobs = function(conf, reg) { runOSCommandLinux("qselect", "-u $USER -s EHQRTW")$output }
staged.queries <- TRUE # To avoid file system locks, cf. [1]
and ~/.BatchJobs,torque.tmpl
is:
## Job name
#PBS -N <%= job.name %>
## Merge standard error and output:
#PBS -j oe
## Direct streams to our logfile:
#PBS -o <%= log.file %>
## Resources needed:
<% if (length(resources) > 0) {
opts <- unlist(resources, use.names=TRUE)
opts <- sprintf("%s=%s", names(opts), opts)
opts <- paste(opts, collapse=",") %>
#PBS -l <%= opts %>
<% } %>
## Run R:
Rscript --verbose "<%= rscript %>"
With new %tweak%
in future we can/will be able to also do:
> library("async")
> plan(batchjobs, backend="~/.BatchJobs.R")
> x %<=% { availableCores() }
> y %<=% { availableCores() } %tweak% list(resources=list(vmem="1gb", nodes="1:ppn=2"))
> x
PBS
1
> y
PBS
2
And with new `%resources% in async, we can do:
> library("async")
> plan(batchjobs, backend="~/.BatchJobs.R")
> x %<=% { availableCores() }
> y %<=% { availableCores() } %resources% list(vmem="1gb", nodes="1:ppn=2")
> x
PBS
1
> y
PBS
2
Add argument
resources
for BatchJobsAsyncTask futures. These resource parameters should be passed along in thesubmitJobs()
call.