`--threads` parameter not working

[angelovangel]

Operating System

Ubuntu 22.04

Other Linux

No response

Workflow Version

v0.5.0

Workflow Execution

Command line

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-clone-validation --fastq fastq_pass --sample_sheet samplesheet.csv --threads 52

Workflow Execution - CLI Execution Profile

standard (default)

What happened?

ERROR ~ Error executing process > 'pipeline:checkIfEnoughReads (1)'

Caused by: Process requirement exceeds available CPUs -- req: 52; avail: 4

The pipeline works with default --threads parameter (4), but trying to increase above 4 leads to this crash. The machine has 102 CPUs. Also, increasing the executor$local cpus = 4 in nextflow.config to a higher number works.

Relevant log output

ERROR ~ Error executing process > 'pipeline:checkIfEnoughReads (1)'

Caused by:
  Process requirement exceeds available CPUs -- req: 52; avail: 4

Command executed:

  STATUS="Failed due to insufficient reads"
  mv per-read-stats.tsv 42-4.stats
  fastcat -s 42-4 -r 42-4.interim -a 2000 -b 6000 input.fastq.gz     | bgzip -@ 51 > interim.fastq.gz
  if [[ "$(wc -l < "42-4.interim")" -ge "48" ]]; then
      mv interim.fastq.gz 42-4.fastq.gz
      STATUS="Completed successfully"
  fi

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /data/projects/2308-plasmid-ricardo/work/96/22a2829f5cdab1115cdf9e64853a03

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
WARN: Killing running tasks (1)

Application activity log entry

No response

[nrhorner]

HI @angelovangel

please see: https://labs.epi2me.io/wfquickstart/

You need to specify the max number of CPUs you would like the workflow in a file like this.

executor {
    $local {
        cpus = 52
        memory = "64 GB"
    }
}

And refer to this file with -c