specifying cpu and mem requirements

[ccrobertson]

When running pepatac with looper, I can’t quite figure out how the number of cpus-per-task and memory requirements are calculated. Is this done within divvy, looper, or pepatac?

I am using the uva_rivanna.yaml config file with the slurm_template.sub as the default template (most recent versions obtained from https://github.com/pepkit/divcfg.git)

If I wanted to reduce the number of cpus requested, could I manually change one of these templates or config files?

[nsheff]

it comes from this tsv file in the pepatac repository:

https://github.com/databio/pepatac/blob/master/resources-sample.tsv

It varies based on input file size. If you just change that file for whatever input file size you need, it should adjust the slurm submission scripts.

[jpsmith5]

And to add to that. If you're submitting with looper, you can override compute settings like so:

looper run <my_config.yaml> --compute mem=8000 cpus-per-task=16 time=00-08:00:00

And so forth...

[nsheff]

if you want to read more about how this works, look at the docs here: http://looper.databio.org/en/latest/pipeline-interface-specification/#size_dependent_variables

[ccrobertson]

Ok great. Thanks for all the pointers! This helps a lot.