search

ablab / spades

SPAdes Genome Assembler
http://ablab.github.io/spades/
Other
756 stars 137 forks source link

SPADES Hybrid assembly #13

Open tangerzhang opened 7 years ago

tangerzhang commented 7 years ago

Dear Developer, I am using SPADES 3.10 for 50X illumina and 10X pacbio assembly. The genome size is 6G and my RAM is 1T. My command line is: spades.py --careful -t 24 -m 980 --dataset dataset.yaml -o SPADES_out

However, I got error below: 7:16:41.441 13G / 13G INFO K-mer Splitting (kmer_data.cpp : 113) Total 1426550508 reads processed 7:16:42.959 96M / 13G INFO General (kmer_index_builder.hpp : 252) Starting k-mer counting. 8:40:59.337 96M / 13G INFO General (kmer_index_builder.hpp : 258) K-mer counting done. There are 24720520342 kmers in total. 8:40:59.337 96M / 13G INFO General (kmer_index_builder.hpp : 260) Merging temporary buckets. 8:48:55.827 96M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 437) Building perfect hash indices 8:48:55.828 96M / 13G WARN K-mer Index Building (kmer_index_builder.hpp : 451) Number of threads was limited down to 12 in order to fit the memory limits during the index 9:28:28.780 7G / 462G INFO General (kmer_index_builder.hpp : 276) Merging final buckets. 9:34:04.745 7G / 462G INFO K-mer Index Building (kmer_index_builder.hpp : 483) Index built. Total 8076657616 bytes occupied (2.61375 bits per kmer). 9:34:04.747 7G / 462G ERROR K-mer Counting (kmer_data.cpp : 356) The reads contain too many k-mers to fit into available memory. You need approx. 920.911GB o

== Error == system call for: "['/home/zhangxt/software/SPAdes-3.10.0-Linux/bin/hammer', '/share/bioinfo/zhangxt/project/12_SES208_SPADES/SPADES_out/corrected/configs/config.info']" finishe

======= SPAdes pipeline finished abnormally and WITH WARNINGS!

=== Error correction and assembling warnings:

=== ERRORs:

I though my memory should be enough. Is there any way to solve this problem? Thanks so much!

asl commented 7 years ago

Hello

Will you please attach your spades.log file?

tangerzhang commented 7 years ago

spades.log.txt params.txt

Please see the attachment for spades.log and params.txt! Looking forward to your help! Thanks!

asl commented 7 years ago

Well, the command line is "/home/zhangxt/software/SPAdes-3.10.0-Linux/bin/spades.py --careful -t 24 -m 800 --dataset /share/bioinfo/zhangxt/project/12_SES208_SPADES/dataset.yaml -o /share/bioinfo/zhangxt/project/12_SES208_SPADES/SPADES_out", note that the memory limit was set to 800 Gb.

tangerzhang commented 7 years ago

Hi asl, Thanks for your quick respone! When I tried to set -m 980, I had another error. Please see the attachment: params-2.txt spades-2.log.txt

asl commented 7 years ago

Well, here your system failed to fulfil SPAdes' request for memory allocation. You had 191 Gb of RAM already used by SPAdes, SPAdes tried to allocate 184 more and your OS returned an error.

tangerzhang commented 7 years ago

Then, any suggestion to solve this problem? Which parameter should I modified? Thanks!

2017-02-13 20:12 GMT+08:00 Anton Korobeynikov notifications@github.com:

Well, here your system failed to fulfil SPAdes' request for memory allocation. You had 191 Gb of RAM already used by SPAdes, SPAdes tried to allocate 184 more and your OS returned an error.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ablab/spades/issues/13#issuecomment-279370531, or mute the thread https://github.com/notifications/unsubscribe-auth/AGG2qnQFPx9sE-E3qT5AGSonAoH0XsUjks5rcEiggaJpZM4L846- .

asl commented 7 years ago

The last problem is not a SPAdes' problem, unfortunately. Make sure there is enough free RAM, etc.