HenrikBengtsson / future.batchtools

:rocket: R package future.batchtools: A Future API for Parallel and Distributed Processing using batchtools
https://future.batchtools.futureverse.org
84 stars 9 forks source link

batchtools templates: `resources[["asis"]]` for as-is declarations #78

Open HenrikBengtsson opened 2 years ago

HenrikBengtsson commented 2 years ago

Issue

There is no standard for how batchtools resources are used in the batchtools templates. This means that we all have slightly different templates and ways to declare the resources argument.

Suggestion

I'd like to suggest that we could reserve resources[["asis"]] to mean "use these strings as-is CLI options/declaration for the scheduler". This would work the same for all templates and would allow the user to pass arbitrary options to the scheduler (via declaration comments).

For example,

asis <- c("-pe smp 2", "-R yes")
plan(batchtools_sge, resources = list(asis = asis, mem_free="1G"))

should be result in the following SGE declarations in the template:

#$ -pe smp 2
#$ -R yes
#$ -l mem_free=1G
HenrikBengtsson commented 1 year ago

Maybe "as-is" resource specifications should be even more verbatim, e.g.

asis <- c("#$ -pe smp 2", "#$ -R yes")
resources <- list(asis = asis, mem_free="1G")
plan(batchtools_sge, resources = resources)

Internally, it can be pre-pasted using:

resources[["asis"]] <- paste(c(resources[["asis"]], ""), collapse = "\n")

so it can be used in the templates as:

<%= resources[["as-is"]] %>

to be rendered as:

#$ -pe smp 2
#$ -R yes