Error running Illumina process 'ncovIllumina:sequenceAnalysis:readMapping'

[mikek498]

I've built a docker container for the illumina workflow based on the latest "environment-illumina.yml" and hosted it here: gcr.io/biga-testing/artic-ncov2019-illumina:dev

The workflow runs however it stops when it reaches the ncovIllumina:sequenceAnalysis:readMapping process and it outputs the following errors:

Error executing process > 'ncovIllumina:sequenceAnalysis:readMapping (IDR2000-011191-01_S2_L001)'

Caused by:

Process ncovIllumina:sequenceAnalysis:readMapping (IDR2000-011191-01_S2_L001) terminated with an error exit status (137)

Command executed: bwa mem -t 1 ref.fa IDR2000-011191-01_S2_L001_R1_001_val_1.fq.gz IDR2000-011191-01_S2_L001_R2_001_val_2.fq.gz | samtools view -bS | samtools sort -o IDR2000-011191-01_S2_L001.sorted.bam

Command exit status: 137 Command output: (empty)

The error text has: [M::mem_pestat] skip orientation RF as there are not enough pairs [M::mem_pestat] skip orientation RR as there are not enough pairs .command.sh: line 2: 17 Broken pipe bwa mem -t 1 ref.fa IDR2000-011191-01_S2_L001_R1_001_val_1.fq.gz IDR2000-011191-01_S2_L001_R2_001_val_2.fq.gz 18 | samtools view -bS 19 Killed | samtools sort -o IDR2000-011191-01_S2_L001.sorted.bam

[m-bull]

Exit 137 is usually a result of a process being killed by the kernel because its using too much memory.

If you're just running this locally in Docker, you could try adding some memory allocation to that process by:

1. Creating file local_resources.conf with the following content:

process {
    withLabel: largecpu {
        cpus = 4
        memory = { 2.GB * task.attempt }
        errorStrategy { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
        maxRetries 3
    }
}

This will re-try the process up to 3 times (if it gets OOM killed), increasing the requested memory allocation by 2GB each time. Adjust this to fit your local resources - the above should work fine on a machine with upto 8GB of RAM. If you have more you can increase memory.

2. Including this config in your nextflow run command: nextflow run -c /path/to/local_resources.conf ncov2019-artic-nf ...

[mikek498]

Increasing memory for various processes worked by adding memory '24 GB' . It was running in a docker container on Google Compute.