search

ablab / spades

SPAdes Genome Assembler
http://ablab.github.io/spades/
Other
752 stars 135 forks source link

I have error with Spades for assembling viral sequence #1370

Closed lqhungpt closed 1 month ago

lqhungpt commented 1 month ago

Description of bug

spades -1 '/home/hungluong/20240905_prrsv_illumina/final5/209656_R1.fastq.gz' -2 '/home/hungluong/20240905_prrsv_illumina/final5/209656_R2.fastq.gz' -o test209656_0912_spade Command line: /usr/lib/spades/bin/spades.py -1 /home/hungluong/20240905_prrsv_illumina/final5/209656_R1.fastq.gz -2 /home/hungluong/20240905_prrsv_illumina/final5/209656_R2.fastq.gz -o /home/hungluong/test209656_0912_spade

System information: SPAdes version: 3.13.1 Python version: 3.10.12 OS: Linux-6.8.0-40-generic-x86_64-with-glibc2.35

Output dir: /home/hungluong/test209656_0912_spade Mode: read error correction and assembling Debug mode is turned OFF

Dataset parameters: Multi-cell mode (you should set '--sc' flag if input data was obtained with MDA (single-cell) technology or --meta flag if processing metagenomic dataset) Reads: Library number: 1, library type: paired-end orientation: fr left reads: ['/home/hungluong/20240905_prrsv_illumina/final5/209656_R1.fastq.gz'] right reads: ['/home/hungluong/20240905_prrsv_illumina/final5/209656_R2.fastq.gz'] interlaced reads: not specified single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/hungluong/test209656_0912_spade/tmp Threads: 16 Memory limit (in Gb): 31

======= SPAdes pipeline started. Log can be found here: /home/hungluong/test209656_0912_spade/spades.log

===== Read error correction started.

== Running read error correction tool: /usr/lib/spades/bin/spades-hammer /home/hungluong/test209656_0912_spade/corrected/configs/config.info

0:00:00.000 4M / 4M INFO General (main.cpp : 75) Starting BayesHammer, built from N/A, git revision N/A 0:00:00.000 4M / 4M INFO General (main.cpp : 76) Loading config from /home/hungluong/test209656_0912_spade/corrected/configs/config.info 0:00:00.000 4M / 4M INFO General (main.cpp : 78) Maximum # of threads to use (adjusted due to OMP capabilities): 4 0:00:00.000 4M / 4M INFO General (memory_limit.cpp : 49) Memory limit set to 31 Gb 0:00:00.000 4M / 4M INFO General (main.cpp : 86) Trying to determine PHRED offset 0:00:00.000 4M / 4M INFO General (main.cpp : 92) Determined value is 33 0:00:00.000 4M / 4M INFO General (hammer_tools.cpp : 36) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ] 0:00:00.000 4M / 4M INFO General (main.cpp : 113) Size of aux. kmer data 24 bytes === ITERATION 0 begins === 0:00:00.001 4M / 4M INFO K-mer Index Building (kmer_index_builder.hpp : 301) Building kmer index 0:00:00.001 4M / 4M INFO General (kmer_index_builder.hpp : 117) Splitting kmer instances into 64 files using 4 threads. This might take a while. 0:00:00.001 4M / 4M INFO General (file_limit.hpp : 32) Open file limit set to 1024 0:00:00.001 4M / 4M INFO General (kmer_splitters.hpp : 89) Memory available for splitting buffers: 2.58301 Gb 0:00:00.001 4M / 4M INFO General (kmer_splitters.hpp : 97) Using cell size of 1048576 0:00:00.001 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 97) Processing /home/hungluong/20240905_prrsv_illumina/final5/209656_R1.fastq.gz 0:00:12.474 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 1199710 reads 0:00:24.516 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 2401272 reads 0:00:36.318 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 3613627 reads 0:00:48.767 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 4826502 reads 0:01:00.602 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 6034253 reads 0:01:13.428 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 7248817 reads 0:01:26.375 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 8433706 reads 0:01:39.373 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 9633743 reads 0:01:51.616 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 10824046 reads 0:02:51.430 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 16927094 reads 0:02:59.532 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 97) Processing /home/hungluong/20240905_prrsv_illumina/final5/209656_R2.fastq.gz 0:05:40.232 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 107) Processed 34344348 reads 0:05:52.040 3G / 3G INFO K-mer Splitting (kmer_data.cpp : 112) Total 35527260 reads processed 0:05:52.160 16M / 3G INFO General (kmer_index_builder.hpp : 120) Starting k-mer counting. 0:08:02.266 16M / 3G INFO General (kmer_index_builder.hpp : 127) K-mer counting done. There are 1592606366 kmers in total. 0:08:02.266 16M / 3G INFO General (kmer_index_builder.hpp : 133) Merging temporary buckets. 0:08:27.992 16M / 3G INFO K-mer Index Building (kmer_index_builder.hpp : 314) Building perfect hash indices 0:11:05.689 780M / 3G INFO General (kmer_index_builder.hpp : 150) Merging final buckets. 0:11:36.659 780M / 3G INFO K-mer Index Building (kmer_index_builder.hpp : 336) Index built. Total 738510800 bytes occupied (3.7097 bits per kmer). 0:11:36.665 780M / 3G ERROR K-mer Counting (kmer_data.cpp : 351) The reads contain too many k-mers to fit into available memory. You need approx. 59.3292GB of free RAM to assemble your dataset

== Error == system call for: "['/usr/lib/spades/bin/spades-hammer', '/home/hungluong/test209656_0912_spade/corrected/configs/config.info']" finished abnormally, err code: 255

In case you have troubles running SPAdes, you can write to spades.support@cab.spbu.ru or report an issue on our GitHub repository github.com/ablab/spades Please provide us with params.txt and spades.log files from the output directory.

spades.log

spades.log

params.txt

params.txt

SPAdes version

3.13.1

Operating System

Ubuntu 20.04.4

Python Version

No response

Method of SPAdes installation

Manual

No errors reported in spades.log

asl commented 1 month ago

The log clearly reads:

0:11:36.665 780M / 3G ERROR K-mer Counting (kmer_data.cpp : 351) The reads contain too many k-mers to fit into available memory. You need approx. 59.3292GB of free RAM to assemble your dataset

Please let us know if this message is not clear enough.