about the bug in nextdenovo2.0

wangzhongkai1 commented 4 years ago

Dear Hu, Thanks for your excellent work in denovo genome assembly of nextdenovo-2.0 and it truly helped us a lot in our project; but i found a bug while running another genome assembly in nextdenovo-2.0

I have finished step 1 of raw align ; the main program was at substep 1, get cns, of step 2 , cns align, and several sub tasks were submitted into running; most of them(26/30) had been normally finished, but some (4/30) were trapped into sleeping state; while I checked the results, I found that the size of cns.fasta was basically the same as other finished ones, and the total number of sequence was 9584, but the corresponding index was imcomplete with only 9490 lines representing the sequence information and the last line had only a number "2" which was the begining of the sequence id; unfortunately, while I killed the main program and manually submitted the trapped subtask, it still could not normally finish and the results were exactly the same as the one I described before.

Could you please help me debug this problem? Thanks a lot, and looking forward to your reply!

Bestwishes!

moold commented 4 years ago

Could you paste the get cns log in here?

wangzhongkai1 commented 4 years ago

hostname

hostname cd /share/home/ryd/xyjj/01_nextdenovo-2.0/01_rundir/02.cns_align/01.get_cns.sh.work/get_cns00
cd /share/home/ryd/xyjj/01_nextdenovo-2.0/01_rundir/02.cns_align/01.get_cns.sh.work/get_cns00 time python /share/home/ryd/bin/NextDenovo-2.0/lib/nextcorrect.py -f /share/home/ryd/xyjj/01_nextdenovo-2.0/./01_rundir/02.cns_align//01.get_cns.input.idxs -i /share/home/ryd/xyjj/01_nextdenovo-2.0/01_rundir/01.raw_align/03.sort_align.sh.work/sort_align00/input.seed.013.sorted.ovl -p 8 -max_lq_length 10000 -o cns.fasta;
python /share/home/ryd/bin/NextDenovo-2.0/lib/nextcorrect.py -f /share/home/ryd/xyjj/01_nextdenovo-2.0/./01_rundir/02.cns_align//01.get_cns.input.idxs -i /share/home/ryd/xyjj/01_nextdenovo-2.0/01_rundir/01.raw_align/03.sort_align.sh.work/sort_align00/input.seed.013.sorted.ovl -p 8 -max_lq_length 10000 -o cns.fasta [INFO] 2019-10-28 11:40:37,316 Corrected step options: [INFO] 2019-10-28 11:40:37,317 Namespace(blacklist='/share/home/ryd/xyjj/01_nextdenovo-2.0/01_rundir/01.raw_align/03.sort_align.sh.work/sort_align00/input.seed.013.sorted.ovl.bl', dbuf=False, fast=False, idxs='/share/home/ryd/xyjj/01_nextdenovo-2.0/./01_rundir/02.cns_align//01.get_cns.input.idxs', max_cov_aln=130, max_lq_length=10000, min_cov_base=4, min_cov_seed=10, min_error_corrected_ratio=0.8, min_len_aln=500, min_len_seed=10000, out='cns.fasta', ovl='/share/home/ryd/xyjj/01_nextdenovo-2.0/01_rundir/01.raw_align/03.sort_align.sh.work/sort_align00/input.seed.013.sorted.ovl', process=8, split=False) [WARNING] 2019-10-28 11:40:37,397 Skip 47571 seeds in blacklist. [INFO] 2019-10-28 11:40:37,797 Start a cns worker in 19860 from parent 19849 [INFO] 2019-10-28 11:40:37,798 Start a cns worker in 19864 from parent 19849 [INFO] 2019-10-28 11:40:37,798 Start a cns worker in 19862 from parent 19849 [INFO] 2019-10-28 11:40:37,799 Start a cns worker in 19866 from parent 19849 [INFO] 2019-10-28 11:40:37,800 Start a cns worker in 19867 from parent 19849 [INFO] 2019-10-28 11:40:37,801 Start a cns worker in 19869 from parent 19849 [INFO] 2019-10-28 11:40:37,802 Start a cns worker in 19873 from parent 19849 [INFO] 2019-10-28 11:40:37,803 Start a cns worker in 19871 from parent 19849 [INFO] 2019-10-28 14:37:27,045 Start a cns worker in 20875 from parent 19849

moold commented 4 years ago

The log shows a thread has crashed maybe because of a known bug, I plan to release the next version and fix it in this week.

wangzhongkai1 commented 4 years ago

Bingo, thank you.

zijiangyang commented 4 years ago

@moold Hi Hu: I got the same error on one project but the nextdenovo works perfectly for several other genome assembly projects. The settings in run.cfg for these projects are exactly the same. May I ask is this error related to the input data?

Many thanks!

moold commented 4 years ago

In some extent, yes. This bug will be triggered only if a seed contains lots of low quality regions. I will fix it in the next release.

zijiangyang commented 4 years ago

In some extent, yes. This bug will be triggered only if a seed contains lots of low quality regions. I will fix it in the next release.

That would be great! Really appreciate your effort in developing this amazing software!

moold commented 4 years ago

Hi, you can try to download the latest version and this bug has been fixed.

wangzhongkai1 commented 4 years ago

Dear Hu, Glad to tell tou that the program has been normally finished, thanks for your effort!

Nextomics / NextDenovo

about the bug in nextdenovo2.0 #21