heathsc / gemBS-rs

A re-write of the gemBS pipeline framework in Rust
BSD 3-Clause "New" or "Revised" License
4 stars 1 forks source link

`jobs` parameter has no effect #12

Open bounlu opened 2 years ago

bounlu commented 2 years ago

Below is my config file. I tried setting jobs parameter outside of steps as well as inside of calling and extract steps. But still it has no effect as they are run one at a time rather than parallelized as expected. I am running the pipeline on a 64 vCPUs instance.

How can I make it work?

base = "/home"

index_dir = ${base}/reference
sequence_dir = ${base}/fastq
bam_dir = ${base}/mapping
bcf_dir = ${base}/calling
extract_dir = ${base}/extract
report_dir = ${base}/report

# hg38 genome index prepared in advance
reference = ${base}/reference/genome.fasta
index = ${base}/reference/genome.gemBS.gem

jobs = 8
cores = 32
threads = 64
memory = 256G

[dbsnp]

# dbsnp index prepared in advance
dbsnp_files = ${base}/reference/dbsnp_146.hg38.vcf.gz
dbsnp_index = ${base}/reference/dbsnp_146.hg38.gemBS.vcf.idx
dbsnp_type = "VCF"

[mapping]

non_stranded = False
remove_individual_bams = True

[calling]

jobs = 8
mapq_threshold = 10
qual_threshold = 13
reference_bias = 2
left_trim = 5
right_trim = 0
keep_improper_pairs = False
keep_duplicates = False
haploid = False
conversion = 0.01,0.05
remove_individual_bcfs = True
contig_pool_limit = 25000000

[extract]

jobs = 8
strand_specific = True
bigwig_strand_specific = True
phred_threshold = 10
make_cpg = True
make_non_cpg = True
make_bedmethyl = True
make_bigwig = True
make_snps = True