search

ablab / spades

SPAdes Genome Assembler
http://ablab.github.io/spades/
Other
763 stars 139 forks source link

system call for: "['/opt/conda/envs/assembly/bin/spades-hammer', '/data/work/DNBSEQ/data1/SPAdes/corrected/configs/config.info']" finished abnormally, OS return value: 21 #1415

Closed xiaoxinganlin closed 1 week ago

xiaoxinganlin commented 1 week ago

Description of bug

== Error == system call for: "['/opt/conda/envs/assembly/bin/spades-hammer', '/data/work/DNBSEQ/data1/SPAdes/corrected/configs/config.info']" finished abnormally, OS return value: 21 None

In case you have troubles running SPAdes, you can report an issue on our GitHub repository github.com/ablab/spades Please provide us with params.txt and spades.log files from the output directory.

SPAdes log can be found here: /data/work/DNBSEQ/data1/SPAdes/spades.log

Thank you for using SPAdes! If you use it in your research, please cite:

Prjibelski, A., Antipov, D., Meleshko, D., Lapidus, A. and Korobeynikov, A., 2020. Using SPAdes de novo assembler. Current protocols in bioinformatics, 70(1), p.e102.

spades.log

spades.py -1 /data/work/DNBSEQ/data1/1.T21_1.fastq.gz -2 /data/work/DNBSEQ/data1/2.T21_2.fastq.gz -o /data/work/DNBSEQ/data1/SPAdes /opt/conda/envs/assembly/share/spades/spades_pipeline/support.py:488: SyntaxWarning: invalid escape sequence '\d' return [atoi(c) for c in re.split("(\d+)", text)]

== Warning == No assembly mode was specified! If you intend to assemble high-coverage multi-cell/isolate data, use '--isolate' option.

Command line: /opt/conda/envs/assembly/bin/spades.py -1 /data/work/DNBSEQ/data1/1.T21_1.fastq.gz -2 /data/work/DNBSEQ/data1/2.T21_2.fastq.gz -o /data/work/DNBSEQ/data1/SPAdes

System information: SPAdes version: 4.0.0 Python version: 3.13.0 OS: Linux-5.15.0-43-generic-x86_64-with-glibc2.17

Output dir: /data/work/DNBSEQ/data1/SPAdes Mode: read error correction and assembling Debug mode is turned OFF

Dataset parameters: Standard mode For multi-cell/isolate data we recommend to use '--isolate' option; for single-cell MDA data use '--sc'; for metagenomic data use '--meta'; for RNA-Seq use '--rna'. Reads: Library number: 1, library type: paired-end orientation: fr left reads: ['/data/work/DNBSEQ/data1/1.T21_1.fastq.gz'] right reads: ['/data/work/DNBSEQ/data1/2.T21_2.fastq.gz'] interlaced reads: not specified single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Assembly graph output will use GFA v1.2 format Other parameters: Dir for temp files: /data/work/DNBSEQ/data1/SPAdes/tmp Threads: 16 Memory limit (in Gb): 250

======= SPAdes pipeline started. Log can be found here: /data/work/DNBSEQ/data1/SPAdes/spades.log

/data/work/DNBSEQ/data1/1.T21_1.fastq.gz: max reads length: 100 /data/work/DNBSEQ/data1/2.T21_2.fastq.gz: max reads length: 100

Reads length: 100

===== Before start started.

===== Read error correction started.

===== Read error correction started.

== Running: /opt/conda/envs/assembly/bin/spades-hammer /data/work/DNBSEQ/data1/SPAdes/corrected/configs/config.info

0:00:00.000 1M / 23M INFO General (main.cpp : 76) Starting BayesHammer, built from N/A, git revision N/A 0:00:00.021 1M / 23M INFO General (main.cpp : 77) Loading config from "/data/work/DNBSEQ/data1/SPAdes/corrected/configs/config.info" 0:00:00.041 1M / 23M INFO General (main.cpp : 79) Maximum # of threads to use (adjusted due to OMP capabilities): 16 0:00:00.041 1M / 23M INFO General (memory_limit.cpp : 55) Memory limit set to 250 Gb 0:00:00.042 1M / 23M INFO General (main.cpp : 87) Trying to determine PHRED offset 0:00:00.044 1M / 23M INFO General (main.cpp : 93) Determined value is 33 0:00:00.044 1M / 23M INFO General (hammer_tools.cpp : 40) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ] 0:00:00.044 1M / 23M INFO General (main.cpp : 114) Size of aux. kmer data 24 bytes === ITERATION 0 begins === 0:00:00.074 1M / 23M INFO General (kmer_index_builder.hpp : 258) Splitting kmer instances into 16 files using 16 threads. This might take a while. 0:00:00.080 1M / 23M INFO General (file_limit.hpp : 43) Open file limit set to 1048576 0:00:00.080 1M / 23M INFO General (kmer_splitter.hpp : 94) Memory available for splitting buffers: 5.20833 Gb 0:00:00.080 1M / 23M INFO General (kmer_splitter.hpp : 102) Using cell size of 4194304 0:00:00.082 9217M / 9217M INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" 0:01:04.361 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 5898518 reads 0:02:09.983 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 11266539 reads 0:03:23.148 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 16826642 reads 0:03:38.739 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 18333888 reads 0:03:38.739 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" 0:03:56.237 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 23080999 reads 0:04:16.132 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 28373020 reads 0:04:36.697 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 34109745 reads 0:04:46.134 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 36665961 reads 0:04:46.135 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 113) Total 36665961 reads processed 0:04:46.718 1M / 15G INFO General (kmer_index_builder.hpp : 264) Starting k-mer counting. 0:05:16.060 1M / 15G INFO General (kmer_index_builder.hpp : 275) K-mer counting done. There are 271487006 kmers in total. 0:05:16.070 1M / 15G INFO K-mer Index Building (kmer_index_builder.hpp : 410) Building perfect hash indices 0:05:43.101 204M / 15G INFO K-mer Index Building (kmer_index_builder.hpp : 446) Index built. Total 271487006 kmers, 196097064 bytes occupied (5.77846 bits per kmer). 0:05:43.101 204M / 15G INFO K-mer Counting (kmer_data.cpp : 355) Arranging kmers in hash map order mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2176843776 bytes, address: 0x7f9381954000, alignment: 67108864, commit: 1) mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2176843776 bytes, address: 0x7f92fe400000, alignment: 67108864, commit: 1) 0:05:48.548 4348M / 15G INFO General (main.cpp : 149) Clustering Hamming graph. 0:08:21.551 4348M / 15G INFO General (main.cpp : 156) Extracting clusters: 0:08:21.551 4348M / 15G INFO General (concurrent_dsu.cpp : 19) Connecting to root 0:08:22.031 4348M / 15G INFO General (concurrent_dsu.cpp : 35) Calculating counts mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2285895680 bytes, address: 0x7f9273c00000, alignment: 67108864, commit: 1) 0:08:49.747 6524M / 15G INFO General (concurrent_dsu.cpp : 64) Writing down entries mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2176843776 bytes, address: 0x7f91ee400000, alignment: 67108864, commit: 1) 0:10:25.434 4348M / 15G INFO General (main.cpp : 168) Clustering done. Total clusters: 82264988 0:10:25.635 2276M / 15G INFO K-mer Counting (kmer_data.cpp : 372) Collecting K-mer information, this takes a while. mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (6522142720 bytes, address: 0x7f91fb400000, alignment: 67108864, commit: 1) 0:10:28.535 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 378) Processing "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" 0:11:54.047 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 378) Processing "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" 0:13:19.638 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 385) Collection done, postprocessing. 0:13:21.570 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 398) There are 271487006 kmers in total. Among them 189289172 (69.7231%) are singletons. 0:13:21.570 8492M / 15G INFO General (main.cpp : 174) Subclustering Hamming graph 0:19:27.682 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 637) Subclustering done. Total 4725 non-read kmers were generated. 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 638) Subclustering statistics: 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 639) Total singleton hamming clusters: 31505922. Among them 5852978 (18.5774%) are good 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 640) Total singleton subclusters: 1672620. Among them 1647043 (98.4708%) are good 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 641) Total non-singleton subcluster centers: 56508677. Among them 41043634 (72.6324%) are good 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 642) Average size of non-trivial subcluster: 4.24744 kmers 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 643) Average number of sub-clusters per non-singleton cluster: 1.14622 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 644) Total solid k-mers: 48543655 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 645) Substitution probabilities: ( 0.934038 0.0202679 0.0177246 0.0279696 ) ( 0.0178556 0.95278 0.0128314 0.0165331 ) ( 0.0163694 0.0127652 0.95289 0.017975 ) ( 0.0278814 0.017679 0.0202349 0.934205 ) 0:19:27.796 8492M / 15G INFO General (main.cpp : 179) Finished clustering. 0:19:27.796 8492M / 15G INFO General (main.cpp : 198) Starting solid k-mers expansion in 16 threads. 0:20:41.037 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 0 produced 9576266 new k-mers. 0:22:01.817 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 1 produced 990515 new k-mers. 0:23:16.964 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 2 produced 56496 new k-mers. 0:24:40.500 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 3 produced 4487 new k-mers. 0:26:03.234 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 4 produced 437 new k-mers. 0:28:21.525 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 5 produced 38 new k-mers. 0:29:32.778 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 6 produced 0 new k-mers. 0:29:32.778 8492M / 15G INFO General (main.cpp : 223) Solid k-mers finalized 0:29:32.779 8492M / 15G INFO General (hammer_tools.cpp : 226) Starting read correction in 16 threads. 0:29:32.779 8492M / 15G INFO General (hammer_tools.cpp : 239) Correcting pair of reads: "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" and "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" 0:29:37.477 9619M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 0 of 1600000 reads. 0:29:40.516 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 0 0:29:42.604 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 0 0:29:46.313 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 1 of 1600000 reads. 0:29:49.308 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 1 0:29:51.414 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 1 0:29:55.603 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 2 of 1600000 reads. 0:29:59.242 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 2 0:30:01.333 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 2 0:30:05.186 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 3 of 1600000 reads. 0:30:08.276 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 3 0:30:10.291 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 3 0:30:14.114 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 4 of 1600000 reads. 0:30:17.229 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 4 0:30:22.627 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 4 0:30:27.495 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 5 of 1600000 reads. 0:30:30.602 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 5 0:30:32.657 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 5 0:30:36.498 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 6 of 1600000 reads. 0:30:39.556 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 6 0:30:41.912 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 6 0:30:45.757 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 7 of 1600000 reads. 0:30:48.865 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 7 0:30:53.242 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 7 0:30:57.079 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 8 of 1600000 reads. 0:31:00.547 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 8 0:31:02.615 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 8 0:31:06.446 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 9 of 1600000 reads. 0:31:09.919 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 9 0:31:12.176 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 9 0:31:16.357 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 10 of 1600000 reads. 0:31:19.839 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 10 0:31:25.626 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 10 0:31:27.860 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 11 of 732073 reads. 0:31:29.503 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 11 0:31:30.450 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 11 0:31:30.451 9716M / 15G ERROR General (hammer_tools.cpp : 194) Pair of read files "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" and "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" contain unequal amount of reads === Stack Trace === /opt/conda/envs/assembly/bin/spades-hammer(+0x1c5b7) [0x55cadf62f5b7] /opt/conda/envs/assembly/bin/spades-hammer(+0x2b086) [0x55cadf63e086] /opt/conda/envs/assembly/bin/spades-hammer(+0x2f04c) [0x55cadf64204c] /opt/conda/envs/assembly/bin/spades-hammer(+0x17022) [0x55cadf62a022] /usr/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f942cfcd555] /opt/conda/envs/assembly/bin/spades-hammer(+0x1af1a) [0x55cadf62df1a]

params.txt

spades.py -1 /data/work/DNBSEQ/data1/1.T21_1.fastq.gz -2 /data/work/DNBSEQ/data1/2.T21_2.fastq.gz -o /data/work/DNBSEQ/data1/SPAdes /opt/conda/envs/assembly/share/spades/spades_pipeline/support.py:488: SyntaxWarning: invalid escape sequence '\d' return [atoi(c) for c in re.split("(\d+)", text)]

== Warning == No assembly mode was specified! If you intend to assemble high-coverage multi-cell/isolate data, use '--isolate' option.

Command line: /opt/conda/envs/assembly/bin/spades.py -1 /data/work/DNBSEQ/data1/1.T21_1.fastq.gz -2 /data/work/DNBSEQ/data1/2.T21_2.fastq.gz -o /data/work/DNBSEQ/data1/SPAdes

System information: SPAdes version: 4.0.0 Python version: 3.13.0 OS: Linux-5.15.0-43-generic-x86_64-with-glibc2.17

Output dir: /data/work/DNBSEQ/data1/SPAdes Mode: read error correction and assembling Debug mode is turned OFF

Dataset parameters: Standard mode For multi-cell/isolate data we recommend to use '--isolate' option; for single-cell MDA data use '--sc'; for metagenomic data use '--meta'; for RNA-Seq use '--rna'. Reads: Library number: 1, library type: paired-end orientation: fr left reads: ['/data/work/DNBSEQ/data1/1.T21_1.fastq.gz'] right reads: ['/data/work/DNBSEQ/data1/2.T21_2.fastq.gz'] interlaced reads: not specified single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Assembly graph output will use GFA v1.2 format Other parameters: Dir for temp files: /data/work/DNBSEQ/data1/SPAdes/tmp Threads: 16 Memory limit (in Gb): 250

======= SPAdes pipeline started. Log can be found here: /data/work/DNBSEQ/data1/SPAdes/spades.log

/data/work/DNBSEQ/data1/1.T21_1.fastq.gz: max reads length: 100 /data/work/DNBSEQ/data1/2.T21_2.fastq.gz: max reads length: 100

Reads length: 100

===== Before start started.

===== Read error correction started.

===== Read error correction started.

== Running: /opt/conda/envs/assembly/bin/spades-hammer /data/work/DNBSEQ/data1/SPAdes/corrected/configs/config.info

0:00:00.000 1M / 23M INFO General (main.cpp : 76) Starting BayesHammer, built from N/A, git revision N/A 0:00:00.021 1M / 23M INFO General (main.cpp : 77) Loading config from "/data/work/DNBSEQ/data1/SPAdes/corrected/configs/config.info" 0:00:00.041 1M / 23M INFO General (main.cpp : 79) Maximum # of threads to use (adjusted due to OMP capabilities): 16 0:00:00.041 1M / 23M INFO General (memory_limit.cpp : 55) Memory limit set to 250 Gb 0:00:00.042 1M / 23M INFO General (main.cpp : 87) Trying to determine PHRED offset 0:00:00.044 1M / 23M INFO General (main.cpp : 93) Determined value is 33 0:00:00.044 1M / 23M INFO General (hammer_tools.cpp : 40) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ] 0:00:00.044 1M / 23M INFO General (main.cpp : 114) Size of aux. kmer data 24 bytes === ITERATION 0 begins === 0:00:00.074 1M / 23M INFO General (kmer_index_builder.hpp : 258) Splitting kmer instances into 16 files using 16 threads. This might take a while. 0:00:00.080 1M / 23M INFO General (file_limit.hpp : 43) Open file limit set to 1048576 0:00:00.080 1M / 23M INFO General (kmer_splitter.hpp : 94) Memory available for splitting buffers: 5.20833 Gb 0:00:00.080 1M / 23M INFO General (kmer_splitter.hpp : 102) Using cell size of 4194304 0:00:00.082 9217M / 9217M INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" 0:01:04.361 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 5898518 reads 0:02:09.983 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 11266539 reads 0:03:23.148 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 16826642 reads 0:03:38.739 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 18333888 reads 0:03:38.739 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" 0:03:56.237 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 23080999 reads 0:04:16.132 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 28373020 reads 0:04:36.697 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 34109745 reads 0:04:46.134 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 36665961 reads 0:04:46.135 9217M / 15G INFO K-mer Splitting (kmer_data.cpp : 113) Total 36665961 reads processed 0:04:46.718 1M / 15G INFO General (kmer_index_builder.hpp : 264) Starting k-mer counting. 0:05:16.060 1M / 15G INFO General (kmer_index_builder.hpp : 275) K-mer counting done. There are 271487006 kmers in total. 0:05:16.070 1M / 15G INFO K-mer Index Building (kmer_index_builder.hpp : 410) Building perfect hash indices 0:05:43.101 204M / 15G INFO K-mer Index Building (kmer_index_builder.hpp : 446) Index built. Total 271487006 kmers, 196097064 bytes occupied (5.77846 bits per kmer). 0:05:43.101 204M / 15G INFO K-mer Counting (kmer_data.cpp : 355) Arranging kmers in hash map order mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2176843776 bytes, address: 0x7f9381954000, alignment: 67108864, commit: 1) mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2176843776 bytes, address: 0x7f92fe400000, alignment: 67108864, commit: 1) 0:05:48.548 4348M / 15G INFO General (main.cpp : 149) Clustering Hamming graph. 0:08:21.551 4348M / 15G INFO General (main.cpp : 156) Extracting clusters: 0:08:21.551 4348M / 15G INFO General (concurrent_dsu.cpp : 19) Connecting to root 0:08:22.031 4348M / 15G INFO General (concurrent_dsu.cpp : 35) Calculating counts mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2285895680 bytes, address: 0x7f9273c00000, alignment: 67108864, commit: 1) 0:08:49.747 6524M / 15G INFO General (concurrent_dsu.cpp : 64) Writing down entries mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (2176843776 bytes, address: 0x7f91ee400000, alignment: 67108864, commit: 1) 0:10:25.434 4348M / 15G INFO General (main.cpp : 168) Clustering done. Total clusters: 82264988 0:10:25.635 2276M / 15G INFO K-mer Counting (kmer_data.cpp : 372) Collecting K-mer information, this takes a while. mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (6522142720 bytes, address: 0x7f91fb400000, alignment: 67108864, commit: 1) 0:10:28.535 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 378) Processing "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" 0:11:54.047 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 378) Processing "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" 0:13:19.638 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 385) Collection done, postprocessing. 0:13:21.570 8492M / 15G INFO K-mer Counting (kmer_data.cpp : 398) There are 271487006 kmers in total. Among them 189289172 (69.7231%) are singletons. 0:13:21.570 8492M / 15G INFO General (main.cpp : 174) Subclustering Hamming graph 0:19:27.682 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 637) Subclustering done. Total 4725 non-read kmers were generated. 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 638) Subclustering statistics: 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 639) Total singleton hamming clusters: 31505922. Among them 5852978 (18.5774%) are good 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 640) Total singleton subclusters: 1672620. Among them 1647043 (98.4708%) are good 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 641) Total non-singleton subcluster centers: 56508677. Among them 41043634 (72.6324%) are good 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 642) Average size of non-trivial subcluster: 4.24744 kmers 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 643) Average number of sub-clusters per non-singleton cluster: 1.14622 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 644) Total solid k-mers: 48543655 0:19:27.683 8492M / 15G INFO Hamming Subclustering (kmer_cluster.cpp : 645) Substitution probabilities: ( 0.934038 0.0202679 0.0177246 0.0279696 ) ( 0.0178556 0.95278 0.0128314 0.0165331 ) ( 0.0163694 0.0127652 0.95289 0.017975 ) ( 0.0278814 0.017679 0.0202349 0.934205 ) 0:19:27.796 8492M / 15G INFO General (main.cpp : 179) Finished clustering. 0:19:27.796 8492M / 15G INFO General (main.cpp : 198) Starting solid k-mers expansion in 16 threads. 0:20:41.037 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 0 produced 9576266 new k-mers. 0:22:01.817 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 1 produced 990515 new k-mers. 0:23:16.964 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 2 produced 56496 new k-mers. 0:24:40.500 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 3 produced 4487 new k-mers. 0:26:03.234 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 4 produced 437 new k-mers. 0:28:21.525 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 5 produced 38 new k-mers. 0:29:32.778 8492M / 15G INFO General (main.cpp : 219) Solid k-mers iteration 6 produced 0 new k-mers. 0:29:32.778 8492M / 15G INFO General (main.cpp : 223) Solid k-mers finalized 0:29:32.779 8492M / 15G INFO General (hammer_tools.cpp : 226) Starting read correction in 16 threads. 0:29:32.779 8492M / 15G INFO General (hammer_tools.cpp : 239) Correcting pair of reads: "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" and "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" 0:29:37.477 9619M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 0 of 1600000 reads. 0:29:40.516 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 0 0:29:42.604 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 0 0:29:46.313 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 1 of 1600000 reads. 0:29:49.308 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 1 0:29:51.414 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 1 0:29:55.603 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 2 of 1600000 reads. 0:29:59.242 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 2 0:30:01.333 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 2 0:30:05.186 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 3 of 1600000 reads. 0:30:08.276 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 3 0:30:10.291 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 3 0:30:14.114 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 4 of 1600000 reads. 0:30:17.229 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 4 0:30:22.627 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 4 0:30:27.495 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 5 of 1600000 reads. 0:30:30.602 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 5 0:30:32.657 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 5 0:30:36.498 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 6 of 1600000 reads. 0:30:39.556 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 6 0:30:41.912 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 6 0:30:45.757 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 7 of 1600000 reads. 0:30:48.865 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 7 0:30:53.242 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 7 0:30:57.079 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 8 of 1600000 reads. 0:31:00.547 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 8 0:31:02.615 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 8 0:31:06.446 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 9 of 1600000 reads. 0:31:09.919 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 9 0:31:12.176 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 9 0:31:16.357 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 10 of 1600000 reads. 0:31:19.839 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 10 0:31:25.626 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 10 0:31:27.860 9716M / 15G INFO General (hammer_tools.cpp : 173) Prepared batch 11 of 732073 reads. 0:31:29.503 9716M / 15G INFO General (hammer_tools.cpp : 180) Processed batch 11 0:31:30.450 9716M / 15G INFO General (hammer_tools.cpp : 190) Written batch 11 0:31:30.451 9716M / 15G ERROR General (hammer_tools.cpp : 194) Pair of read files "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" and "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" contain unequal amount of reads === Stack Trace === /opt/conda/envs/assembly/bin/spades-hammer(+0x1c5b7) [0x55cadf62f5b7] /opt/conda/envs/assembly/bin/spades-hammer(+0x2b086) [0x55cadf63e086] /opt/conda/envs/assembly/bin/spades-hammer(+0x2f04c) [0x55cadf64204c] /opt/conda/envs/assembly/bin/spades-hammer(+0x17022) [0x55cadf62a022] /usr/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f942cfcd555] /opt/conda/envs/assembly/bin/spades-hammer(+0x1af1a) [0x55cadf62df1a]

SPAdes version

SPA

Operating System

windows

Python Version

pyhthon

Method of SPAdes installation

conda

No errors reported in spades.log

asl commented 1 week ago

The log clearly reads:

0:31:30.451 9716M / 15G ERROR General (hammer_tools.cpp : 194) Pair of read files "/data/work/DNBSEQ/data1/1.T21_1.fastq.gz" and "/data/work/DNBSEQ/data1/2.T21_2.fastq.gz" contain unequal amount of reads