search

voutcn / megahit

Ultra-fast and memory-efficient (meta-)genome assembler
http://www.ncbi.nlm.nih.gov/pubmed/25609793
GNU General Public License v3.0
595 stars 135 forks source link

Assembly takes too much time for a specific k-mer size. #181

Closed gelomerase closed 6 years ago

gelomerase commented 6 years ago

I am running a coassembly of three paired end sets (six files) and I seem to always get stuck on the SdBG assembly at k57. I also tried a previous assembly using default parameters and got stuck at k59.

MEGAHIT v1.1.3 --- [Fri Aug 17 11:48:58 2018] Start assembly. Number of CPU threads 16 --- --- [Fri Aug 17 11:48:58 2018] Available memory: 540852912128, used: 486767620915 --- [Fri Aug 17 11:48:58 2018] Converting reads to binaries --- /usr/local/bin/megahit_asm_core buildlib B1megahitassembly_kmer2/tmp/reads.lib B1megahitassembly_kmer2/tmp/reads.lib b' [read_lib_functions-inl.h : 209] Lib 0 (B1R1_P_qtrim15.fq,B1R2_P_qtrim15.fq): pe, 75712598 reads, 150 max length' b' [read_lib_functions-inl.h : 209] Lib 1 (B2R1_P_qtrim15.fq,B2R2_P_qtrim15.fq): pe, 83371902 reads, 150 max length' b' [read_lib_functions-inl.h : 209] Lib 2 (B3R1_P_qtrim15.fq,B3R2_P_qtrim15.fq): pe, 81137888 reads, 150 max length' b' [utils.h : 126] Real: 423.9410\tuser: 361.7685\tsys: 56.8442\tmaxrss: 164868' --- [Fri Aug 17 11:56:07 2018] k list: 27,37,47,57,67,77,87,97,107,117,127 --- --- [Fri Aug 17 11:56:07 2018] Extracting solid (k+1)-mers for k = 27 --- cmd: /usr/local/bin/megahit_sdbg_build count -k 27 -m 2 --host_mem 486767620915 --mem_flag 1 --gpu_mem 0 --output_prefix B1megahitassembly_kmer2/tmp/k27/27 --num_cpu_threads 16 --num_output_threads 5 --readlib file B1megahitassembly_kmer2/tmp/reads.lib

I have set the job to run for 72 hours and it gets terminated at k57/59 during the "Unitigs removed in excessive pruning..." step and it seems that no error is being produced except for job termination due to time.

I'm really not sure where I am encountering an error...am I running out of memory?

voutcn commented 6 years ago

You Just my guess, it was running out disk space. You can see the memory utilization of Megahit by linux tools like htop or top. Should you have further questions please reopen this issue and attach to full log here.