BatchJobs

NOTE: Development continues in the new package batchtools

Provides Map, Reduce and Filter variants to generate jobs on batch computing systems like PBS/Torque, LSF and Sun Grid Engine. Multicore and SSH systems are also supported.

• Official CRAN release site: https://cran.r-project.org/package=BatchJobs

• R Documentation in HTML: https://www.rdocumentation.org/packages/BatchJobs

• Run this in R to install the current GitHub version:

  devtools::install_github("tudo-r/BatchJobs")

• Further installation instructions

If you use the package, please cite it and star it here

JSS link: https://www.jstatsoft.org/v64/i11

To cite BatchJobs or BatchExperiments in publications use:

  Bernd Bischl, Michel Lang, Olaf Mersmann, Jörg Rahnenführer, Claus Weihs (2015). BatchJobs and BatchExperiments: Abstraction Mechanisms for Using R in Batch Environments. Journal
  of Statistical Software, 64(11), 1-25. URL https://www.jstatsoft.org/v64/i11/.

A BibTeX entry for LaTeX users is

  @Article{,
    title = {{BatchJobs} and {BatchExperiments}: Abstraction Mechanisms for Using {R} in Batch Environments},
    author = {Bernd Bischl and Michel Lang and Olaf Mersmann and J{\"o}rg Rahnenf{\"u}hrer and Claus Weihs},
    journal = {Journal of Statistical Software},
    year = {2015},
    volume = {64},
    number = {11},
    pages = {1--25},
    url = {https://www.jstatsoft.org/v64/i11/},
  }

Core features

• Create, submit and control R jobs on batch systems
• Provides the functional programming tools Map, Reduce and Filter to operate on the cluster
• Supported systems include Torque/PBS, SGE, SLURM, OpenLava and LSF
• Support for makeshift SSH clusters and local (multicore) execution
• Convenient collection and aggregation of results
• Further Map and Reduce results from previous jobs as batch jobs
• Optional mail sending using sendmailR after job completion
• Query status of jobs and display log files inside R
• Possibility to write your own simple cluster interface if your architecture is not supported
• BatchExperiments extends this package with functionality required for comprehensive computer experiments and simulation studies.

Configuration

A fresh installation defaults to a local execution mode. Proceed to Configuration to set up cluster execution. The package also allows a few R options to be set, see here.

Documentation

Probably the best way to get started in 5 minutes is to read our paper or Henrik Bengtsson's intro slides. If you have more time, read our technical report. For more detailed information on the functions consult the R documentation We also provide a FAQ in our wiki. And here is a nice intro article for BatchJobs with Torque on R Bloggers.

We also have a mailing list.