voutcn / megahit

Ultra-fast and memory-efficient (meta-)genome assembler
http://www.ncbi.nlm.nih.gov/pubmed/25609793
GNU General Public License v3.0
585 stars 134 forks source link

exit code -6 #373

Open 2061574124 opened 2 months ago

2061574124 commented 2 months ago

i have around 60 samples, that performed paired-end seq for metagenome, total ~400G clean data. i have combine all forward reads and reverse reads, gained F.fastq,gz and R.fastq.gz. and than i run:

megahit -1 /home/dell/raw_data/F.fastq.gz -2 /home/dell/raw_data/R.fastq.gz -o megahit_out --out-prefix assembly -m 0.5 -t 25 94.313Gb memory in total. Using: 47.157Gb. MEGAHIT v1.1.3 --- [Thu Jun 27 18:05:02 2024] Start assembly. Number of CPU threads 25 --- --- [Thu Jun 27 18:05:02 2024] Available memory: 101267902464, used: 50633951232 --- [Thu Jun 27 18:05:02 2024] Converting reads to binaries --- b' [read_lib_functions-inl.h : 209] Lib 0 (/home/dell/raw_data/F.fastq.gz,/home/dell/raw_data/R.fastq.gz): pe, 5694767552 reads, 150 max length' b' [utils.h : 126] Real: 9029.7895\tuser: 4891.5591\tsys: 609.5392\tmaxrss: 150828' --- [Thu Jun 27 20:35:32 2024] k list: 21,29,39,59,79,99,119,141 --- --- [Thu Jun 27 20:35:32 2024] Extracting solid (k+1)-mers for k = 21 --- Error occurs when running "sdbg_builder count/read2sdbg", please refer to megahit_out/assembly.log for detail [Exit code -6]

my computer memory is 96G, and ~1T space available. So I want to know the specific reason for the termination, and if there is any good analysis process recommended based on the existing computer? Looking forward to your reply very much Thanks

zuozuozuozuozuozuozuo commented 1 month ago

I encountered an identical error when running the megahit combined in metawrap as follows:

MEGAHIT v1.1.3 --- [Sat Jul 13 10:15:28 2024] Start assembly. Number of CPU threads 40 --- --- [Sat Jul 13 10:15:28 2024] Available memory: 404272803840, used: 150000000000 --- [Sat Jul 13 10:15:28 2024] k list: 21,29,39,59,79,99,119,141 --- --- [Sat Jul 13 10:15:28 2024] Building graph for k = 21 --- Error occurs when running "builder build" for k = 21; please refer to /mnt/16T_1/zuo/ASSEMBLY/L23/megahit/log for detail [Exit code -6]

From my present perspective, I think this error derives from the disk storage. I am not sure, still waiting for the same answer with you.

Good luck we too.

davidbio commented 1 month ago

I found this https://github.com/voutcn/megahit/issues/225#issuecomment-514917881.

You can find N and n in the log file, here is an arbitrary example.

I believe this is n: 2024-07-22 13:08:02 - b'INFO sorting/kmer_counter.cpp : 406 - Total number of candidate reads: 405260170 (963210911)'

And this could be N: 2024-07-22 13:08:16 - b'INFO sorting/seq_to_sdbg.cpp : 415 - Bases to reserve: 1764619673100, number contigs: 0, number multiplicity: 58820655770'

Which leads me to (I think M does not matter.): (1764619673100 / 4) + (16 * 405260170) = 447639080995

Which equals 447 GB.

Correct me if I am wrong.