multi-process parameter not working

[tkmamidi]

I'm working with slurm on a HPC and couldn't get the --mp parameter working. Here's the command I used and the error:

oc run testing_variants_hg38.vcf.gz -l hg38 --mp 20 -t csv --package mypackage

Also, it is picking the available CPUs on the node and not from the slurm job resources. maybe the num_workers should be modified to reflect situations with HPC usage?

Any help is appreciated!

Thanks!

[tkmamidi]

Something to consider while getting the number of CPUs available for n_workers - https://stackoverflow.com/a/55423170

[kmoad]

Probably a good idea to use os.sched_getaffinity on linux systems. We'll look into adding that for the next release.