raw align error

[Jianz9293]

Describe the bug Hello, the following problem occurred when running NextDenovo(v2.4.0)

Error message Paste the complete log message, include the main task log and failed subtask log. The main task log is usually located in your working directory and is named pidXXX.log.info and the main task log will tell you the failed subtask log in the last few lines, such as:

[INFO] 2021-07-14 04:09:19,490 Submit jobID:[21476] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align57/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 04:09:50,986 Submit jobID:[21509] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align58/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 04:10:01,419 Submit jobID:[21526] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align59/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:45,364 52 raw_align jobs failed, and rerun for the 3 time
[INFO] 2021-07-14 05:15:45,370 Total jobs: 52
[INFO] 2021-07-14 05:15:45,372 Submit jobID:[25394] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:45,875 Submit jobID:[25399] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align01/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:46,378 Submit jobID:[25405] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align02/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:46,881 Submit jobID:[25410] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align03/nextDenovo.sh] in the local_cycle.

Genome characteristics `genome size 200M

Input data Types Count (#) Bases (bp) Depth (X) Raw 2842058 18982058099 94.91 Clean 2842058 18982058099 94.91 Count (#) Length (bp) N50 955354 6868

Config file [General] job_type = local # local, slurm, sge, pbs, lsf job_prefix = nextDenovo task = all # all, correct, assemble rewrite = yes # yes/no deltmp = yes parallel_jobs = 20 # number of tasks used to run in parallel input_type = raw # raw, corrected read_type = ont # clr, ont, hifi input_fofn = input.fofn workdir = 01_rundir

[correct_option] read_cutoff = 1k genome_size = 200m # estimated genome size sort_options = -m 20g -t 15 seed_depth = 25 minimap2_options_raw = -x ava-ont -t 8 pa_correction = 3 # number of corrected tasks used to run in parallel, each corrected task requires ~TOTAL_INPUT_BASES/4 bytes of memory usage. correction_options = -p 10

[assemble_option] random_round = 20 minimap2_options_cns = -t 8 nextgraph_options = -a 1

Operating system no found, linux

GCC gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)

Python Python 2.7.18

NextDenovo v2.4.0

To Reproduce (Optional) Steps to reproduce the behavior. Providing a minimal test dataset on which we can reproduce the behavior will generally lead to quicker turnaround time!

Additional context (Optional) Add any other context about the problem here.

[moold]

Could you paste the content of /home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh.e to here?

[Jianz9293]

Thank you for your reply,After I read through other similar problems, I thought it was due to lack of memory, so I ran the same program again on a disk with enough memory, but the same problem still occurred. Now I write out the information of the wrong file here is /home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh.e hostname

• hostname cd /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00
• cd /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00 time /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl;
• /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh: line 5: 363 Killed /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl

And /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh: line 5 is here

!/bin/bash

set -xve hostname cd /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00 time /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl; touch /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh.done

[moold]

The system killed subtaskes (see /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh: line 5: 363 Killed), which usually caused by lack of memory, see here to optimize parallel computing parameters.

[Jianz9293]

OK, I will adjust the parameters and try again. Thank you very much for your reply

[moold]

enen... I will close this issue but feel free to re-open if you still have this issue.