search

ablab / spades

SPAdes Genome Assembler
http://ablab.github.io/spades/
Other
763 stars 139 forks source link

err code: -11 #58

Open cistarsa opened 6 years ago

cistarsa commented 6 years ago

Now I'm using the whole data set, as opposed to a subset, and getting err code -11. Thank you and is there a way that I could look these up?

Command line: ./spades.py   --pe1-1 /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_180bp_male_1CLEAN.fq    --pe1-2 /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_180bp_male_2CLEAN.fq    --pe2-1 /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_500bp_male_1_CLEAN.fq   --pe2-2 /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_500bp_male_2_CLEAN.fq   --mp1-1 /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec3Kb_male_1_CLEAN.fq  --mp1-2 /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec3Kb_male_2_CLEAN.fq  --only-assembler    -t  8   -k  37  --trusted-contigs   /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_redundans_on_alpaths_contigs_genome.fa  --trusted-contigs   /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Redundans_jelly.out.fasta    --nanopore  /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Minion_CPB.fq    -o  /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222  

System information:
  SPAdes version: 3.11.1
  Python version: 2.7.5
  OS: Linux-3.10.0-693.11.1.el7.centos.plus.x86_64-x86_64-with-centos-7.4.1708-Core

Output dir: /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222
Mode: ONLY assembling (without read error correction)
Debug mode is turned OFF

Dataset parameters:
  Multi-cell mode (you should set '--sc' flag if input data was obtained with MDA (single-cell) technology or --meta flag if processing metagenomic dataset)
  Reads:
    Library number: 1, library type: mate-pairs
      orientation: rf
      left reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec3Kb_male_1_CLEAN.fq']
      right reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec3Kb_male_2_CLEAN.fq']
      interlaced reads: not specified
      single reads: not specified
    Library number: 2, library type: paired-end
      orientation: fr
      left reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_180bp_male_1CLEAN.fq']
      right reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_180bp_male_2CLEAN.fq']
      interlaced reads: not specified
      single reads: not specified
    Library number: 3, library type: paired-end
      orientation: fr
      left reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_500bp_male_1_CLEAN.fq']
      right reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_500bp_male_2_CLEAN.fq']
      interlaced reads: not specified
      single reads: not specified
    Library number: 4, library type: nanopore
      left reads: not specified
      right reads: not specified
      interlaced reads: not specified
      single reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Minion_CPB.fq']
    Library number: 5, library type: trusted-contigs
      left reads: not specified
      right reads: not specified
      interlaced reads: not specified
      single reads: ['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_redundans_on_alpaths_contigs_genome.fa', '/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Redundans_jelly.out.fasta']
Assembly parameters:
  k: [37]
  Repeat resolution is enabled
  Mismatch careful mode is turned OFF
  MismatchCorrector will be SKIPPED
  Coverage cutoff is turned OFF
Other parameters:
  Dir for temp files: /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222/tmp
  Threads: 8
  Memory limit (in Gb): 109

======= SPAdes pipeline started. Log can be found here: /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222/spades.log

== Processing additional contigs (/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Ldec_redundans_on_alpaths_contigs_genome.fa): changing Ns to As and splitting by continues (>= 10) Ns fragments (results are in /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222/split_input directory)
== Processing additional contigs (/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Redundans_jelly.out.fasta): changing Ns to As and splitting by continues (>= 10) Ns fragments (results are in /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222/split_input directory)

===== Assembling started.

== Running assembler: K37

  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  76)   Loading config from /var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222/K37/configs/config.info
  0:00:00.000     4M / 4M    INFO    General                 (memory_limit.hpp          :  51)   Memory limit set to 109 Gb
  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  89)   Starting SPAdes, built from N/A, git revision N/A
  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  90)   Maximum k-mer length: 128
  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  91)   Assembling dataset (/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222/dataset.info) with K=37
  0:00:00.000     4M / 4M    INFO    General                 (launch.hpp                :  51)   SPAdes started
  0:00:00.000     4M / 4M    INFO    General                 (launch.hpp                :  58)   Starting from stage: construction
  0:00:00.000     4M / 4M    INFO    General                 (launch.hpp                :  61)   Two-step RR enabled: 0
  0:00:00.002     4M / 4M    INFO    General                 (launch.hpp                :  79)   Will need read mapping, kmer mapper will be attached
  0:00:00.002     4M / 4M    INFO   StageManager             (stage.cpp                 : 126)   STAGE == Construction
  0:00:00.008     4M / 4M    INFO    General                 (construction.cpp          :  57)   Trusted contigs will be used in graph construction
  0:00:00.013     4M / 4M    INFO    General                 (read_converter.hpp        :  84)   Converting reads to binary format for library #1 (takes a while)
  0:00:00.013     4M / 4M    INFO    General                 (read_converter.hpp        :  85)   Converting paired reads
  0:00:00.340    80M / 80M   INFO    General                 (binary_converter.hpp      : 139)   16384 reads processed
  0:00:00.596    88M / 88M   INFO    General                 (binary_converter.hpp      : 139)   32768 reads processed
  0:00:01.094   112M / 112M  INFO    General                 (binary_converter.hpp      : 139)   65536 reads processed
  0:00:01.856   152M / 152M  INFO    General                 (binary_converter.hpp      : 139)   131072 reads processed
  0:00:03.612   240M / 240M  INFO    General                 (binary_converter.hpp      : 139)   262144 reads processed
  0:00:07.905   408M / 408M  INFO    General                 (binary_converter.hpp      : 139)   524288 reads processed
  0:00:16.445   676M / 676M  INFO    General                 (binary_converter.hpp      : 139)   1048576 reads processed
  0:00:31.866   676M / 676M  INFO    General                 (binary_converter.hpp      : 139)   2097152 reads processed
  0:01:02.094   676M / 676M  INFO    General                 (binary_converter.hpp      : 139)   4194304 reads processed
  0:02:02.170   676M / 676M  INFO    General                 (binary_converter.hpp      : 139)   8388608 reads processed
  0:04:02.114   676M / 676M  INFO    General                 (binary_converter.hpp      : 139)   16777216 reads processed
  0:09:02.619   676M / 680M  INFO    General                 (binary_converter.hpp      : 139)   33554432 reads processed
  0:19:29.799   676M / 680M  INFO    General                 (binary_converter.hpp      : 139)   67108864 reads processed
  0:25:40.945   172M / 680M  INFO    General                 (binary_converter.hpp      : 159)   86751720 reads written
  0:25:41.077     4M / 680M  INFO    General                 (read_converter.hpp        :  94)   Converting single reads
  0:25:41.205   132M / 680M  INFO    General                 (binary_converter.hpp      : 159)   0 reads written
  0:25:41.213     4M / 680M  INFO    General                 (read_converter.hpp        :  84)   Converting reads to binary format for library #2 (takes a while)
  0:25:41.213     4M / 680M  INFO    General                 (read_converter.hpp        :  85)   Converting paired reads
  0:25:41.497    80M / 680M  INFO    General                 (binary_converter.hpp      : 139)   16384 reads processed
  0:25:41.719    88M / 680M  INFO    General                 (binary_converter.hpp      : 139)   32768 reads processed
  0:25:42.152   112M / 680M  INFO    General                 (binary_converter.hpp      : 139)   65536 reads processed
  0:25:43.118   152M / 680M  INFO    General                 (binary_converter.hpp      : 139)   131072 reads processed
  0:25:44.743   236M / 680M  INFO    General                 (binary_converter.hpp      : 139)   262144 reads processed
  0:25:48.294   404M / 680M  INFO    General                 (binary_converter.hpp      : 139)   524288 reads processed
  0:25:57.232   668M / 680M  INFO    General                 (binary_converter.hpp      : 139)   1048576 reads processed
  0:26:14.664   672M / 680M  INFO    General                 (binary_converter.hpp      : 139)   2097152 reads processed
  0:26:46.744   688M / 688M  INFO    General                 (binary_converter.hpp      : 139)   4194304 reads processed
  0:27:51.661   700M / 700M  INFO    General                 (binary_converter.hpp      : 139)   8388608 reads processed
  0:30:01.594   712M / 712M  INFO    General                 (binary_converter.hpp      : 139)   16777216 reads processed
  0:33:56.855   700M / 712M  INFO    General                 (binary_converter.hpp      : 139)   33554432 reads processed
  0:34:04.351   144M / 712M  INFO    General                 (binary_converter.hpp      : 159)   34508653 reads written
  0:34:04.413     4M / 712M  INFO    General                 (read_converter.hpp        :  94)   Converting single reads
  0:34:04.525   132M / 712M  INFO    General                 (binary_converter.hpp      : 159)   0 reads written
  0:34:04.566     4M / 712M  INFO    General                 (graph_construction.hpp    : 120)   Constructing DeBruijn graph for k=37
  0:34:04.597     4M / 712M  INFO    General                 (kmer_index_builder.hpp    : 106)   Splitting kmer instances into 64 buckets using 8 threads. This might take a while.
  0:34:04.597     4M / 712M  INFO    General                 (file_limit.hpp            :  32)   Open file limit set to 16384
  0:34:04.597     4M / 712M  INFO    General                 (kmer_splitters.hpp        :  74)   Memory available for splitting buffers: 4.5415 Gb
  0:34:04.597     4M / 712M  INFO    General                 (kmer_splitters.hpp        :  82)   Using cell size of 524288
  0:34:45.542     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 8326811 reads
  0:35:27.674     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 16652928 reads
  0:36:12.745     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 24973886 reads
  0:36:57.271     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 33289637 reads
  0:37:42.774     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 41602464 reads
  0:38:29.006     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 49909359 reads
  0:39:14.032     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 58219374 reads
  0:40:01.410     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 66546098 reads
  0:40:48.957     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 74863370 reads
  0:41:36.239     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 83182360 reads
  0:42:22.714     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 91490039 reads
  0:43:09.250     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 99793458 reads
  0:47:14.257     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 141386805 reads
  1:01:16.016     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 291)   Processed 274154081 reads
  1:24:59.842     6G / 6G    INFO    General                 (kmer_splitters.hpp        : 297)   Adding contigs from previous K

== Error ==  system call for: "['/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/spades', '/var/lib/condor/execute/slot1/dir_3953/SPAdes-3.11.1-Linux/bin/Hybrid_Assembly1222/K37/configs/config.info']" finished abnormally, err code: -11
fjlicht commented 6 years ago

Similar error here. 42Mb Fungal genome utilizing @tseemann shovill pipeline. Please see attached log file. Is this due to not enough memory being allocated? Failure appears at kmer 111. spades.log params.txt

fjlicht commented 6 years ago

Issue was not enough RAM allocated, fixed with --ram 16, consider solved.