HenrikBengtsson / future.BatchJobs

:rocket: R package: future.BatchJobs: A Future API for Parallel and Distributed Processing using BatchJobs [Intentionally archived on CRAN on 2021-01-08]
https://cran.r-project.org/package=future.BatchJobs
8 stars 0 forks source link

Implement separate batchjobs future constructors #61

Closed HenrikBengtsson closed 8 years ago

HenrikBengtsson commented 8 years ago

Implement separate batchjobs future constructors, e.g.

## Generic (fully compatible with BatchJobs)
plan(batchjobs) ## for .BatchJobs.R
plan(batchjobs, ".BatchJobs.R")

## Template-based setups (with default template files)
plan(batchjobs_torque)
plan(batchjobs_torque, template="torque.tmpl")
plan(batchjobs_slurm, template="slurm.tmpl")
plan(batchjobs_sge, template="sge.tmpl")
plan(batchjobs_lsf, template="lsf.tmpl")

## Built-in setups
plan(batchjobs_interactive)
plan(batchjobs_local)
plan(batchjobs_multicore, workers=4)
plan(batchjobs_ssh, workers=nodes)

(Also, since plan(batchjobs) is basically the only real usage of future.BatchJobs, it won't change much of the existing code. The other types of BatchJobs futures are mostly there for proof-of-concept since they are already supported by the core futures in the future package.)

This is actually much more in line with what's in the future package, cf.

plan(transparent)
plan(eager)
plan(multicore, workers=4)
plan(cluster, workers=nodes)
HenrikBengtsson commented 8 years ago

This far we have:

plan(batchjobs_local)
plan(batchjobs_interactive)
plan(batchjobs_multicore)
plan(batchjobs_multicore, workers=n)
plan(batchjobs_conf)  ## Replaces backend(".BatchJobs.R")
plan(batchjobs_conf, pathnames=pathnames)   ## Replaces backend(pathname[1])