search

xiaochuanle / NECAT

Nanopore data assembler
137 stars 24 forks source link

cns_1_al_vol_0.sh: line 16: 35702 Bus error #57

Open QichengMa opened 1 year ago

QichengMa commented 1 year ago

Dear Chuan-Le,

When I run "necat.pl correct my_config.txt", I got the error message:

"cns_1_al_vol_0.sh: line 16: 35702 Bus error

... ...

/home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh: line 16: 35702 Bus error /home/kggx609/software/NECAT/Linux-amd64/bin/oc2pmov -a 1000 -b 2000 -k 15 -t 1 -q 500 -i 1 -d 0.25 -n 500 -z 10 -m 500 -j 0 -s 3 -u 1 -e 0.5 -t 1 /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData 0 /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData/pm_result_02023-08-03 01:53:33 [Warning] Failed to run script, 135, /home/kggx609/projects/gr1_gfp/fastq_pass/gr1gfp/scripts/cns 1_al_vol_0.sh 2023-08-03 01:53:33 [Error] Reached to maximum number of script errors "

I appreciate if you could help me to configure out where I did wrong.

Thanks,

lemene commented 1 year ago

Hi, @QichengMa

I read your mail and GENOME_SIZE was set to 7000 and PREP_OUTPUT_COVERAGE was set to 40. We haven't tested on such a small genome before. You can try increasing the two parameters, for example, PREP_OUTPUT_COVERAGE=200.

Is it convenient to provide the file size in 1-consensus/cns_iter1?

QichengMa commented 1 year ago

I increased the values, but I still get the same error:

(base) [kggx609@seskscpn075 fastq_pass]$ more my_config.txt PROJECT=gr1_gfp ONT_READ_LIST=read_list.txt GENOME_SIZE=600000 THREADS=1 MIN_READ_LENGTH=1500 PREP_OUTPUT_COVERAGE=200 OVLP_FAST_OPTIONS=-n 500 -z 20 -b 2000 -e 0.5 -j 0 -u 1 -a 1000 OVLP_SENSITIVE_OPTIONS=-n 500 -z 10 -e 0.5 -j 0 -u 1 -a 1000 CNS_FAST_OPTIONS=-a 2000 -x 4 -y 12 -l 1000 -e 0.5 -p 0.8 -u 0 CNS_SENSITIVE_OPTIONS=-a 2000 -x 4 -y 12 -l 1000 -e 0.5 -p 0.8 -u 0 TRIM_OVLP_OPTIONS=-n 100 -z 10 -b 2000 -e 0.5 -j 1 -u 1 -a 400 ASM_OVLP_OPTIONS=-n 100 -z 10 -b 2000 -e 0.5 -j 1 -u 0 -a 400 NUM_ITER=2 CNS_OUTPUT_COVERAGE=30 CLEANUP=1 USE_GRID=false GRID_NODE=0 GRID_OPTIONS= SMALL_MEMORY=0 FSA_OL_FILTER_OPTIONS= FSA_ASSEMBLE_OPTIONS= FSA_CTG_BRIDGE_OPTIONS= POLISH_CONTIGS=true (base) [kggx609@seskscpn075 fastq_pass]$

(base) [kggx609@seskscpn075 fastq_pass]$ tail nohup.out 2023-08-03 23:58:20 [Info] Skip filtering reads for consensus for outputs are newer. 2023-08-03 23:58:20 [Info] Start correcting iteratively for cns 1.. 2023-08-03 23:58:20 [Info] Start aliging for cns 1. 2023-08-03 23:58:20 [Info] Skip making vol for cns 1 for outputs are newer. 2023-08-03 23:58:20 [Info] Start aligning vol for cns 1. 2023-08-03 23:58:20 [Info] Parallelly start aligning vol 0 for cns 1. 2023-08-03 23:58:20 [Info] Run script: /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh 2>&1 |tee /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh.log /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh: line 16: 8202 Bus error /home/kggx609/software/NECAT/Linux-amd64/bin/oc2pmov -e 0.5 -m 500 -a 1000 -u 1 -j 0 -k 15 -b 2000 -i 1 -d 0.25 -t 1 -q 500 -s 3 -n 500 -z 10 -t 1 /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData 0 /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData/pm_result_0 2023-08-03 23:58:38 [Warning] Failed to run script, 135, /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh 2023-08-03 23:58:38 [Error] Reached to maximum number of script errors (base) [kggx609@seskscpn075 fastq_pass]$

(base) [kggx609@seskscpn075 fastq_pass]$ cd - /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1 (base) [kggx609@seskscpn075 cns_iter1]$ ls -ltr total 1 drwxrwx--- 2 kggx609 xaz-scp-rd 4096 Aug 3 01:53 PackedData (base) [kggx609@seskscpn075 cns_iter1]$ pwd /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1 (base) [kggx609@seskscpn075 cns_iter1]$ ls PackedData (base) [kggx609@seskscpn075 cns_iter1]$ cd PackedData (base) [kggx609@seskscpn075 PackedData]$ ls reads_info.txt vol0 volume_names.txt (base) [kggx609@seskscpn075 PackedData]$ ls -ltr total 129 -rw-rw---- 1 kggx609 xaz-scp-rd 75091 Aug 3 01:53 vol0 -rw-rw---- 1 kggx609 xaz-scp-rd 92 Aug 3 01:53 volume_names.txt -rw-rw---- 1 kggx609 xaz-scp-rd 4 Aug 3 01:53 reads_info.txt (base) [kggx609@seskscpn075 PackedData]$ more reads_info.txt 1 3 (base) [kggx609@seskscpn075 PackedData]$ more volume_names.txt /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData/vol0 0 3 (base) [kggx609@seskscpn075 PackedData]$ file vol0 vol0: data (base) [kggx609@seskscpn075 PackedData]$

Vol0 is a binary file.

lemene commented 1 year ago

I forgot to mention that the project folder (gr1_gfp) needs to be deleted first.跟读-------- 原始邮件 --------发件人： QichengMa @.>日期： 2023年8月4日周五 早上8:06收件人： xiaochuanle/NECAT @.>抄送： lemene @.>, Comment @.>主 题： Re: [xiaochuanle/NECAT] cns_1_al_vol_0.sh: line 16: 35702 Bus error (Issue #57) I increased the values, but I still get the same error: (base) @. fastq_pass]$ more my_config.txt PROJECT=gr1_gfp ONT_READ_LIST=read_list.txt GENOME_SIZE=600000 THREADS=1 MIN_READ_LENGTH=1500 PREP_OUTPUT_COVERAGE=200 OVLP_FAST_OPTIONS=-n 500 -z 20 -b 2000 -e 0.5 -j 0 -u 1 -a 1000 OVLP_SENSITIVE_OPTIONS=-n 500 -z 10 -e 0.5 -j 0 -u 1 -a 1000 CNS_FAST_OPTIONS=-a 2000 -x 4 -y 12 -l 1000 -e 0.5 -p 0.8 -u 0 CNS_SENSITIVE_OPTIONS=-a 2000 -x 4 -y 12 -l 1000 -e 0.5 -p 0.8 -u 0 TRIM_OVLP_OPTIONS=-n 100 -z 10 -b 2000 -e 0.5 -j 1 -u 1 -a 400 ASM_OVLP_OPTIONS=-n 100 -z 10 -b 2000 -e 0.5 -j 1 -u 0 -a 400 NUM_ITER=2 CNS_OUTPUT_COVERAGE=30 CLEANUP=1 USE_GRID=false GRID_NODE=0 GRID_OPTIONS= SMALL_MEMORY=0 FSA_OL_FILTER_OPTIONS= FSA_ASSEMBLE_OPTIONS= FSA_CTG_BRIDGE_OPTIONS= POLISH_CONTIGS=true (base) @. fastq_pass]$ (base) @. fastq_pass]$ tail nohup.out 2023-08-03 23:58:20 [Info] Skip filtering reads for consensus for outputs are newer. 2023-08-03 23:58:20 [Info] Start correcting iteratively for cns 1.. 2023-08-03 23:58:20 [Info] Start aliging for cns 1. 2023-08-03 23:58:20 [Info] Skip making vol for cns 1 for outputs are newer. 2023-08-03 23:58:20 [Info] Start aligning vol for cns 1. 2023-08-03 23:58:20 [Info] Parallelly start aligning vol 0 for cns 1. 2023-08-03 23:58:20 [Info] Run script: /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh 2>&1 |tee /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh.log /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh: line 16: 8202 Bus error /home/kggx609/software/NECAT/Linux-amd64/bin/oc2pmov -e 0.5 -m 500 -a 1000 -u 1 -j 0 -k 15 -b 2000 -i 1 -d 0.25 -t 1 -q 500 -s 3 -n 500 -z 10 -t 1 /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData 0 /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData/pm_result_0 2023-08-03 23:58:38 [Warning] Failed to run script, 135, /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh 2023-08-03 23:58:38 [Error] Reached to maximum number of script errors (base) @. fastq_pass]$ (base) @. fastq_pass]$ cd - /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1 (base) @. cns_iter1]$ ls -ltr total 1 drwxrwx--- 2 kggx609 xaz-scp-rd 4096 Aug 3 01:53 PackedData (base) @. cns_iter1]$ pwd /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1 (base) @. cns_iter1]$ ls PackedData (base) @. cns_iter1]$ cd PackedData (base) @. PackedData]$ ls reads_info.txt vol0 volume_names.txt (base) @. PackedData]$ ls -ltr total 129 -rw-rw---- 1 kggx609 xaz-scp-rd 75091 Aug 3 01:53 vol0 -rw-rw---- 1 kggx609 xaz-scp-rd 92 Aug 3 01:53 volume_names.txt -rw-rw---- 1 kggx609 xaz-scp-rd 4 Aug 3 01:53 reads_info.txt (base) @. PackedData]$ more reads_info.txt 1 3 (base) @. PackedData]$ more volume_names.txt /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData/vol0 0 3 (base) @. PackedData]$ file vol0 vol0: data (base) @.*** PackedData]$ Vol0 is a binary file.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

QichengMa commented 1 year ago

I deleted it:

$ /bin/rm -rf gr1_gfp nohup.out $ nohup bash tt &

But I still get errors:

Smartmatch is experimental at /home/kggx609/software/NECAT/Linux-amd64/bin/Plgd/Project.pm line 263. 2023-08-04 00:30:25 [Info] Found Slurm, which is /usr/bin/sinfo 2023-08-04 00:30:25 [Info] Start correcting rawreads. 2023-08-04 00:30:25 [Info] Start filtering reads for consensus. 2023-08-04 00:30:25 [Info] Run script: /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_pprr.sh 2>&1 |tee /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_pprr.sh.log 2023-08-04 00:30:25 [INFO] Start 2023-08-04 00:30:25 [INFO] Arguments: cmd = longest ifname = read_list.txt size = 0 block_size = 0 base_size = 120000000 ofname = /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/raw_reads/filtered_raw_reads.fasta thread_size = 1 names = min_length = 1500 genome_size = 0 id2name = discard_illegal_read = 1

2023-08-04 00:30:35 [INFO] length size = 523144 2023-08-04 00:30:35 [INFO] min_length = 2238 2023-08-04 00:30:37 [INFO] End 2023-08-04 00:30:54 [Info] End filtering reads for consensus. 2023-08-04 00:30:54 [Info] Start correcting iteratively for cns 1.. 2023-08-04 00:30:54 [Info] Start aliging for cns 1. 2023-08-04 00:30:54 [Info] Start making vol for cns 1. 2023-08-04 00:30:54 [Info] Run script: /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_mk_vol.sh 2>&1 | tee /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_mk_vol.sh.log file_list: /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/raw_reads/raw_read_list.txt [Fri Aug 4 00:30:55 2023] INFO: pack /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/rawreads/filtered raw_reads.fasta.gz: 49156 reads, 120105871 bps (makedb/main.c, 44) 2023-08-04 00:30:55 [Info] End making vol for cns 1. 2023-08-04 00:30:55 [Info] Start aligning vol for cns 1. 2023-08-04 00:30:55 [Info] Parallelly start aligning vol 0 for cns 1. 2023-08-04 00:30:55 [Info] Run script: /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh 2>&1 |tee /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh.log /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/scripts/cns_1_al_vol_0.sh: line 16: 18375 Killed /ho me/kggx609/software/NECAT/Linux-amd64/bin/oc2pmov -u 1 -z 10 -s 3 -n 500 -k 15 -j 0 -b 2000 -d 0.25 -i 1 -a 1000 -e 0.5 -m 500 -t 1 -q 500 -t 1 /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData 0 /home/kgg x609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData/pm_result_0 2023-08-04 00:33:14 [Warning] Failed to run script, 137, /home/kggx609/projects/gr1_gfp/fastq_pass/gr1gfp/scripts/cns 1_al_vol_0.sh 2023-08-04 00:33:14 [Error] Reached to maximum number of script errors (base) [kggx609@seskscpn075 fastq_pass]$

BUT, this time, the file vol0 is MUCH larger,

(base) [kggx609@seskscpn075 PackedData]$ pwd /home/kggx609/projects/gr1_gfp/fastq_pass/gr1_gfp/1-consensus/cns_iter1/PackedData (base) [kggx609@seskscpn075 PackedData]$ ls -ltr total 32641 -rw-rw---- 1 kggx609 xaz-scp-rd 33418287 Aug 4 00:30 vol0 -rw-rw---- 1 kggx609 xaz-scp-rd 96 Aug 4 00:30 volume_names.txt -rw-rw---- 1 kggx609 xaz-scp-rd 8 Aug 4 00:30 reads_info.txt (base) [kggx609@seskscpn075 PackedData]$

Thanks again!

lemene commented 1 year ago

Hi, @QichengMa I corrected a dataset of similar size, but there were no errors. Have you used NECAT to correct other datasets before, or is this your first time using it? Could you provide the version of NECAT and the operating system?

QichengMa commented 1 year ago

This is the first time that I use NECAT. This is the latest version of NECAT which I downloaded last week. This is the OS version I used: Linux seskscpn084.prim.scp 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux