search

rkajitani / MetaPlatanus

De novo metagenome assembler
GNU General Public License v3.0
13 stars 1 forks source link

How to set memory #11

Closed kangxiongbin closed 2 years ago

kangxiongbin commented 2 years ago

My data illumina 2X250bp 930M; pacbio 695M.

When I use the default value of memory, metaplatanus were crashed.

out.assembleLog:

MetaPlatanus version: 1.3.0 metaplatanus_core assemble -tmp . -c 2 -C 4 -t 30 -m 756 -f /prj/hybrid-assembly/kang/process/hybrid/real_data/nwcs/metaplatanus/r1.fastq /prj/hybrid-assembly/kang/pr>

AVE_READ_LEN=171.854 K = 25, saving kmers from reads...

PROCESS INFORMATION

VmPeak: 199.949 GByte VmHWM: 3.771 GByte 26.54user 162.73system 3:19.48elapsed 94%CPU (0avgtext+0avgdata 203953792maxresident)k 0inputs+1839896outputs (0major+51022101minor)pagefaults 0swaps

When I use 80G memory, metaplatanus were crashed at the second step iterative assembly.

out.iteLog:

MetaPlatanus version: 1.3.0 metaplatanus_core iterate -tmp . -c initial_merged.fa initial_mergedJunctionKmer.fa -k out_kmer_occ.bin -t 30 -m 80 -IP1 /prj/hybrid-assembly/kang/process/hybrid/real>

Error(16): Divide exception!! meta_platanus divide cannot finished correctly Command exited with non-zero status 16 0.58user 0.10system 0:00.54elapsed 127%CPU (0avgtext+0avgdata 21652maxresident)k 0inputs+17776outputs (0major+40912minor)pagefaults 0swaps

rkajitani commented 2 years ago

Sorry to bother you again... If possible, could you please show the log file below? out_intermediate/2_iterative_asm/out1/out.divLog

kangxiongbin commented 2 years ago

Sorry to bother you again... If possible, could you please show the log file below? out_intermediate/2_iterative_asm/out1/out.divLog

My pleasure.

out.divLog:

MetaPlatanus version: 1.3.0 metaplatanus_core divide -t 30 -tmp . -c out1/out_contig.fa -o out1/out -ont /prj/hybrid-assembly/kang/process/hybrid/real_data/nwcs/metaplatanus/long_10x_2.fq

ececuting platanus_b solve_DBG internally ... MetaPlatanus version: 1.3.0 metaplatanus_core solve_DBG -divide_only -o out1/out -t 30 -tmp . -c out1/out_contig.fa -ont /prj/hybrid-assembly/kang/process/hybrid/real_data/nwcs/metaplatanus/long_10x_2.fq -s 32 64 96

Error(3): Error, File fomat exception!! Read file is unknown format. Error(14): Error, SolveDBG exception!! meta_platanus solve_DBG command failed.

rkajitani commented 2 years ago

Thank you. Like issue #9 , I guess the file of long reads (long_10x_2.fq) consists of non-ACGTN characters such as lower cases. Could you please try to correct it as follows? seqkit -u -w0 long_10x_2.fq >cor_long_10x_2.fq If it consists of other non-ACGTN characters, please replace them with "N". seqkit fq2fa long_10x_2.fq | seqkit replace -w0 -s -p "[^ACGTNacgtn]" -r "N" >cor_long_10x_2.fa

For the memory issue, MetaPlatanus attempts to use the specified memory amount (-m), but it sometimes over-uses the amount (possibly memory leaks ...). So, it may be suitable to specify a little less memory than the actual limit (e.g., -m 512).

kangxiongbin commented 2 years ago

Thank you very much! Now it process well.