search

UCSF-Costello-Lab / LG3_Pipeline

The original LG3 pipeline
https://github.com/UCSF-Costello-Lab/LG3_Pipeline
0 stars 0 forks source link

ROBUSTNESS: Make sure of PBS_NUM_PPN #14

Closed HenrikBengtsson closed 6 years ago

HenrikBengtsson commented 6 years ago

The number of parallel threads/cores are hardcoded in the scripts/*.sh files, e.g.

$BWA aln -t 12 ...

It is better/safer not to assume that we have 12 cores available (e.g. what if that is forgotten/changed in the qsub call(*)). Instead, we should use something like:

ncores=${PBS_NUM_PPN:1}
...
$BWA aln -t ${ncores} ...

That will default to single-core processing unless otherwise specified by PBS_NUM_PPN (which is set by qsub).

(*) I actually found one case where 12 cores are requested, but only 8 threads are used. With PBS_NUM_PPN, it would have used all allocated slots.

HenrikBengtsson commented 6 years ago

Ok, I've done this for the places where it's "obvious" how to do it. At least the following remain:

$ grep -E "[-]nct[ ]+[0-9]" *.pbs scripts/*.sh
scripts/Germline.sh:        ### -nct 3 -nt 8 \
scripts/UG.sh:                -nct 3 -nt 8 \

but since it's a combination of -nct and -nt it's not straightforward. Closing for now.