[Bug]: Process requirement exceeds available CPUs

What happened?

I am trying to run this pipeline on a somewhat large 10x genomics 3' single cell library (~70Gb fastq.gz file). The pipeline errors when executing pipeline:process_bams:stringtie with the following message:

Caused by:
  Process requirement exceeds available CPUs -- req: 6; avail: 4

I have set max_threads to 24 on the command line. Here is the command run (with paths in the command and log altered for privacy).

nextflow run epi2me-labs/wf-single-cell \
  -profile singularity \
  -w wf-sc/workspace \
  --fastq path/to/the.fastq.gz \
  --kit_name 3prime \
  --kit_version v3 \
  --expected_cells 5000 \
  --ref_genome_dir path/to/refdata-gex-GRCh38-2020-A \
  --out_dir wf-sc \
  --max_threads 24 \
  --merge_bam

Operating System

ubuntu 20.04

Workflow Execution

Command line

Workflow Execution - EPI2ME Labs Versions

No response

Workflow Execution - CLI Execution Profile

Singularity

Workflow Version

v0.1.8-g7f96b28

Relevant log output

N E X T F L O W  ~  version 22.10.3
Launching `https://github.com/epi2me-labs/wf-single-cell` [ecstatic_torvalds] DSL2 - revision: 7f96b280f1 [master]

WARN: Found unexpected parameters:
* --process_label: singlecell
- Ignore this warning: params.schema_ignore_params = "process_label"

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-single-cell v0.1.8-g7f96b28
--------------------------------------------------------------------------------
Core Nextflow options
  revision        : master
  runName         : ecstatic_torvalds
  containerEngine : singularity
  launchDir       : /path/to/launch/dir
  workDir         : /path/to/work/dir
  projectDir      : /path/to/userdir/.nextflow/assets/epi2me-labs/wf-single-cell
  userName        : johndoe
  profile         : singularity
  configFiles     : /path/to/userdir/.nextflow/assets/epi2me-labs/wf-single-cell/nextflow.config

Input Options
  fastq           : path/to/the.fastq.gz
  ref_genome_dir  : path/to/refdata-gex-GRCh38-2020-A
  kit_config      : /path/to/userdir/.nextflow/assets/epi2me-labs/wf-single-cell/kit_configs.csv
  merge_bam       : true
  expected_cells  : 5000

Output Options
  out_dir         : wf-sc

Advanced options
  max_threads     : 24
  matrix_min_genes: 200
  umap_plot_genes : /path/to/userdir/.nextflow/assets/epi2me-labs/wf-single-cell/umap_plot_genes.csv

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-single-cell for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-single-cell v0.1.8-g7f96b28.
--------------------------------------------------------------------------------
Checking fastq input.
Single file input detected.
[-        ] process > pipeline:getVersions           -
[-        ] process > pipeline:getParams             -
[-        ] process > pipeline:summariseCatChunkR... -

[-        ] process > pipeline:getVersions           -
[-        ] process > pipeline:getParams             -
[-        ] process > pipeline:summariseCatChunkR... -
[-        ] process > pipeline:stranding:call_ada... -
[-        ] process > pipeline:stranding:combine_... -
[-        ] process > pipeline:stranding:summariz... -
[-        ] process > pipeline:align:call_paftools   -
[-        ] process > pipeline:align:get_chrom_sizes -
[-        ] process > pipeline:align:align_to_ref    -
[-        ] process > pipeline:process_bams:split... -
[-        ] process > pipeline:process_bams:get_c... -
[-        ] process > pipeline:process_bams:extra... -
[-        ] process > pipeline:process_bams:gener... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:strin... -
[-        ] process > pipeline:process_bams:align... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:clust... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:annot... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -
...
many lines of output
...
executor >  local (105)
[81/a5d035] process > pipeline:getVersions           [100%] 1 of 1 ✔
[29/9ee7c2] process > pipeline:getParams             [100%] 1 of 1 ✔
[58/8c74b0] process > pipeline:summariseCatChunkR... [100%] 1 of 1 ✔
[88/cebbbb] process > pipeline:stranding:call_ada... [100%] 24 of 24 ✔
[14/bf95c5] process > pipeline:stranding:combine_... [100%] 1 of 1 ✔
[8d/81462f] process > pipeline:stranding:summariz... [100%] 1 of 1 ✔
[bb/453de3] process > pipeline:align:call_paftools   [100%] 1 of 1 ✔
[77/e927f0] process > pipeline:align:get_chrom_sizes [100%] 1 of 1 ✔
[c2/84becc] process > pipeline:align:align_to_ref... [100%] 1 of 1 ✔
[6f/81d1f4] process > pipeline:process_bams:split... [100%] 1 of 1 ✔
[45/40657e] process > pipeline:process_bams:get_c... [100%] 1 of 1 ✔
[8b/141615] process > pipeline:process_bams:extra... [100%] 40 of 40 ✔
[cf/a9e07e] process > pipeline:process_bams:gener... [100%] 1 of 1 ✔
[16/f69d84] process > pipeline:process_bams:assig... [  2%] 1 of 40
[84/5da471] process > pipeline:process_bams:assig... [100%] 25 of 25 ✔
[08/23d384] process > pipeline:process_bams:strin... [100%] 1 of 1, failed: 1
[-        ] process > pipeline:process_bams:align... -
[-        ] process > pipeline:process_bams:assig... -
[-        ] process > pipeline:process_bams:clust... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:annot... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -
Error executing process > 'pipeline:process_bams:stringtie (1)'

Caused by:
  Process requirement exceeds available CPUs -- req: 6; avail: 4

Command executed:

  # Build transcriptome.
  samtools view -h align.bam          | tee >(stringtie -L  -p 6 -G chr.gtf -l stringtie             -o stringtie.gff - )         | samtools fastq -         | tee reads.fastq         | seqkit seq -n > read_order.tsv

  # Get transcriptome sequence
  gffread -g ref_genome.fa -w transcriptome.fa stringtie.gff

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /wf-sc/workspace/08/23d384a9c1454f98ea71604c7b5b86

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

WARN: Killing running tasks (4)

epi2me-labs / wf-single-cell

[Bug]: Process requirement exceeds available CPUs #15