search

nf-core / denovotranscript

A pipeline for de novo transcriptome assembly of paired-end short reads from bulk RNA-seq
https://nf-co.re/denovotranscript/
MIT License
7 stars 3 forks source link

Error in rnaSpades when run from singularity image #19

Open IdoBar opened 1 month ago

IdoBar commented 1 month ago

Description of the bug

rnaSPADES step fails with the following error:

Process `NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:SPADES (pooled_reads)` terminated with an error exit status (12)
....
spades/spades_pipeline/support.py:488: SyntaxWarning: invalid escape sequence '\d'
    return [atoi(c) for c in re.split("(\d+)", text)]

I'm wondering it's a lack of memory or if it's related to this issue that's been fixed in a recent commit, but not in the Singularity image (v4.0.0).

Command used and terminal output

$ nextflow run nf-core/denovotranscript -r 1.0.0 --input Mnova_samplesheet.csv --transrate_reference $WORKDIR/GIU3625_Humpback_whale.transcript.fasta --outdir assembly_results -with-tower -profile apptainer,bunya -c /home/ibar/.nextflow/bunya.config

Error executing process > 'NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:SPADES (pooled_reads)'

Caused by:
  Process `NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:SPADES (pooled_reads)` terminated with an error exit status (12)

Command executed:

  spades.py \
      --rna \
      --threads 12 \
      --memory 72 \
       \
      -1 pooled_reads_1.merged.fastq.gz -2 pooled_reads_2.merged.fastq.gz   \
       \
      -o ./
  mv spades.log pooled_reads.spades.log

  if [ -f scaffolds.fasta ]; then
      mv scaffolds.fasta pooled_reads.scaffolds.fa
      gzip -n pooled_reads.scaffolds.fa
  fi
  if [ -f contigs.fasta ]; then
      mv contigs.fasta pooled_reads.contigs.fa
      gzip -n pooled_reads.contigs.fa
  fi
  if [ -f transcripts.fasta ]; then
      mv transcripts.fasta pooled_reads.transcripts.fa
      gzip -n pooled_reads.transcripts.fa
  fi
  if [ -f soft_filtered_transcripts.fasta ]; then
      mv soft_filtered_transcripts.fasta pooled_reads.soft_filtered_transcripts.fa
      gzip -n pooled_reads.soft_filtered_transcripts.fa
  fi
  if [ -f hard_filtered_transcripts.fasta ]; then
      mv hard_filtered_transcripts.fasta pooled_reads.hard_filtered_transcripts.fa
      gzip -n pooled_reads.hard_filtered_transcripts.fa
  fi
  if [ -f assembly_graph_with_scaffolds.gfa ]; then
      mv assembly_graph_with_scaffolds.gfa pooled_reads.assembly.gfa
      gzip -n pooled_reads.assembly.gfa
  fi

  if [ -f gene_clusters.fasta ]; then
      mv gene_clusters.fasta pooled_reads.gene_clusters.fa
      gzip -n pooled_reads.gene_clusters.fa
  fi

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_DENOVOTRANSCRIPT:DENOVOTRANSCRIPT:SPADES":
      spades: $(spades.py --version 2>&1 | sed -n 's/^.*SPAdes genome assembler v//p')
  END_VERSIONS

Command exit status:
  12

Command output:
    0:16:08.560  6841M / 7645M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 29316762 reads
    0:16:12.854  6841M / 7645M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 39038979 reads
    0:16:17.374  6841M / 7645M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 48782744 reads
    0:16:21.645  6841M / 7647M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 58498936 reads
    0:16:26.038  6841M / 7647M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 68271535 reads
    0:16:30.401  6841M / 7649M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 78073658 reads
    0:16:35.051  6841M / 7649M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 87867580 reads
    0:16:39.744  6841M / 7649M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 97673676 reads
    0:16:48.487  6841M / 7652M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 107471525 reads
    0:16:53.077  6841M / 7652M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 117319145 reads
    0:17:02.054  6841M / 7652M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 136865777 reads
    0:18:05.624  6841M / 7660M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 273186636 reads
    0:20:13.473  6841M / 7660M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 545964005 reads
    0:24:23.811  6841M / 7662M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 1075403253 reads
    0:32:41.601  6841M / 7671M INFO    General                 (kmer_splitters.hpp        : 128)   Processed 2154882891 reads
    0:37:55.337     1M / 7671M INFO    General                 (kmer_splitters.hpp        : 134)   Used 2804507304 reads
    0:37:55.355     1M / 7671M INFO    General                 (kmer_index_builder.hpp    : 264)   Starting k-mer counting.
    0:37:55.364     1M / 7671M ERROR   General                 (mmapped_reader.hpp        :  52)   mmap(2) failed. Reason: Cannot allocate memory. Error code: 12
  === Stack Trace ===
    0:37:55.364     1M / 7671M ERROR   General                 (mmapped_reader.hpp        :  52)   mmap(2) failed. Reason: Cannot allocate memory. Error code: 12
  === Stack Trace ===
  /usr/local/bin/spades-core(+0x61a97) [0x55605b40ca97]
  /usr/local/bin/spades-core(+0x7cae0) [0x55605b427ae0]
  /usr/local/bin/spades-core(+0x2888cd) [0x55605b6338cd]
  /usr/local/bin/spades-core(+0x28a781) [0x55605b635781]
  /usr/local/bin/../lib/libgomp.so.1(+0x186d9) [0x7f61b79976d9]
  /lib/x86_64-linux-gnu/libc.so.6(+0x89134) [0x7f61b753d134]
  /lib/x86_64-linux-gnu/libc.so.6(__clone+0x40) [0x7f61b75bca40]
  /usr/local/bin/spades-core(+0x61a97) [0x55605b40ca97]
  /usr/local/bin/spades-core(+0x7cae0) [0x55605b427ae0]
  /usr/local/bin/spades-core(+0x2888cd) [0x55605b6338cd]
  /usr/local/bin/spades-core(+0x28a781) [0x55605b635781]
  /usr/local/bin/../lib/libgomp.so.1(+0x186d9) [0x7f61b79976d9]
  /lib/x86_64-linux-gnu/libc.so.6(+0x89134) [0x7f61b753d134]
  /lib/x86_64-linux-gnu/libc.so.6(__clone+0x40) [0x7f61b75bca40]

  == Error ==  system call for: "['/usr/local/bin/spades-core', 'K33/configs/config.info', 'K33/configs/mda_mode.info', 'K33/configs/rna_mode.info']" finished abnormally, OS return value: 12
  None

  In case you have troubles running SPAdes, you can report an issue on our GitHub repository github.com/ablab/spades
  Please provide us with params.txt and spades.log files from the output directory.

  SPAdes log can be found here: spades.log

  Thank you for using rnaSPAdes! If you use it in your research, please cite:

    Bushmanova, E., Antipov, D., Lapidus, A. and Prjibelski, A.D., 2019. rnaSPAdes: a de novo transcriptome assembler and its application to RNA-Seq data. GigaScience, 8(9), p.giz100.
    doi.org/10.1093/gigascience/giz100

Command error:
  /usr/local/share/spades/spades_pipeline/support.py:488: SyntaxWarning: invalid escape sequence '\d'
    return [atoi(c) for c in re.split("(\d+)", text)]

Work dir:
  /scratch/project_mnt/S0016/sandbox/OTE14085/Mnova_denovotranscript_assembly/work/db/51e9b16f8fe073849eefcca37e7af1

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

Relevant files

No response

System information

avani-bhojwani commented 2 days ago

I suspect it ran out of memory first, and then ran into the SyntaxWarning. I can increase the memory allocated for this step