Crash during assembly step

[pbrec]

Hi @Reedwarbler,

I am trying to use repdenovo and it crashes on me, always during the assembly step. I already downsized the reads to 20% of the forward reads only and it still happens, so not sure if read no. is the problem here.

Here is my error message:

rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff100/Asm’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/.fa’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/*.fastq’: No such file or directory [bwa_index] Pack FASTA... 0.10 sec [bwa_index] Construct BWT for the packed sequence... [bwa_index] 1.36 seconds elapse. [bwa_index] Update BWT... 0.05 sec [bwa_index] Pack forward-only FASTA... 0.08 sec [bwa_index] Construct SA from BWT and Occ... 0.83 sec [main] Version: 0.7.5a-r405 [main] CMD: /usr/bin/bwa index /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa [main] Real time: 2.556 sec; CPU: 2.432 sec [M::main_mem] read 26467 sequences (6451911 bp)... [main] Version: 0.7.5a-r405 [main] CMD: /usr/bin/bwa mem -a /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa [main] Real time: 6.968 sec; CPU: 6.842 sec [samopen] SAM header is present: 26467 sequences. open: No such file or directory [bam_index_build2] fail to open the BAM file. Index file not found, now create it!!! Index file cannot be created!!! Bamtools ERROR: could not open input BAM file: /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa.itself.sort.bam rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa.itself.bam’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa.itself.bam’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa.itself.bam’: No such file or directory Segmentation fault (core dumped) Segmentation fault (core dumped) [bwa_index] Pack FASTA... 0.00 sec [bwa_index] Construct BWT for the packed sequence... [bwa_index] 0.00 seconds elapse. [bwa_index] Update BWT... 0.00 sec [bwa_index] Pack forward-only FASTA... 0.00 sec [bwa_index] Construct SA from BWT and Occ... 0.00 sec [main] Version: 0.7.5a-r405 [main] CMD: /usr/bin/bwa index /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa [main] Real time: 0.007 sec; CPU: 0.004 sec [main] Version: 0.7.5a-r405 [main] CMD: /usr/bin/bwa mem -a /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa [main] Real time: 0.002 sec; CPU: 0.002 sec [samopen] no @SQ lines in the header. [sam_read1] missing header? Abort! [bam_header_read] EOF marker is absent. The input is probably truncated. open: No such file or directory [bam_index_build2] fail to open the BAM file. Index file not found, now create it!!! Index file cannot be created!!! Bamtools ERROR: could not open input BAM file: /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.itself.sort.bam rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.itself.bam’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.itself.bam’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.itself.bam’: No such file or directory open: No such file or directory [_razf_open] fail to open /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa [fai_build] fail to open the FASTA file /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa open: No such file or directory [_razf_open] fail to open /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa [fai_build] fail to open the FASTA file /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa [bwa_index] fail to open file '/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa' : No such file or directory [E::bwa_idx_load] fail to locate the index files [samopen] no @SQ lines in the header. [sam_read1] missing header? Abort! [bam_header_read] EOF marker is absent. The input is probably truncated. open: No such file or directory [bam_index_build2] fail to open the BAM file. Index file not found, now create it!!! Index file cannot be created!!! Bamtools ERROR: could not open input BAM file: /home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.sort.bam rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.sa’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.pac’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.bwt’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.ann’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.amb’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.bam’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.bam’: No such file or directory rm: cannot remove ‘/home/pbrand/repdenovo/med-PA/cutoff_100/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.bam’: No such file or directory Traceback (most recent call last): File "/home/pbrand/bin/REPdenovo-master/main.py", line 408, in main_func(scommand,sfconfig,sfreads_list) File "/home/pbrand/bin/REPdenovo-master/main.py", line 371, in main_func RM_DUP_BF_MERGE_CUTOFF, RM_DUP_AF_MERGE_CUTOFF) File "/home/pbrand/bin/REPdenovo-master/MergeContigs.py", line 91, in merge_contigs os.rename(foutput,fout_folder+"contigs.fa") OSError: [Errno 2] No such file or directory

Any idea what's going wrong here?

Best, Philipp

[pbrec]

This here is what's in the contigs.fa_no_dup.fa.merged.fa file:

Arrange error! 0 15

Might help.

[simoncchu]

@pbrec Sorry for the late reply. Would you please run "ls -l" under the main folder and then show me the results?

[pbrec]

-rw-rw-r-- 1 pbrand pbrand 2938092 Mar 28 22:08 25mer.temp_contigs.fa -rw-rw-r-- 1 pbrand pbrand 3037695 Mar 28 22:38 35mer.temp_contigs.fa -rw-rw-r-- 1 pbrand pbrand 1838757 Mar 28 23:09 45mer.temp_contigs.fa drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_121467_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_15183_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:08 Asm_25_1897_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_1943479_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_242934_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_30366_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_3795_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_3886958_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_485869_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_60733_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_7591_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_7773916_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:08 Asm_25_948_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:07 Asm_25_971739_24 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:37 Asm_35_1191_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_1219956_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_152494_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_19061_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:37 Asm_35_2382_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_2439913_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_304989_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_38123_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:37 Asm_35_4765_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_4879827_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:38 Asm_35_595_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_609978_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:36 Asm_35_76247_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 22:37 Asm_35_9530_34 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_1047_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_1072704_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_134088_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_16761_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_2095_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_2145409_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_268176_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_33522_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_4190_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:08 Asm_45_4290818_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_536352_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_67044_44 drwxrwxr-x 2 pbrand pbrand 4096 Mar 28 23:09 Asm_45_8380_44 -rw-rw-r-- 1 pbrand pbrand 1689894 Mar 28 23:09 contigs.fa.fai -rw-rw-r-- 1 pbrand pbrand 20 Mar 28 23:10 contigs.fa_no_dup.fa.merged.fa -rw-rw-r-- 1 pbrand pbrand 0 Mar 28 23:10 contigs.fa_no_dup.fa.merge.info -rw-rw-r-- 1 pbrand pbrand 43033822 Mar 28 22:07 dumped_25mers.txt -rw-rw-r-- 1 pbrand pbrand 40940871 Mar 28 22:36 dumped_35mers.txt -rw-rw-r-- 1 pbrand pbrand 37290253 Mar 28 23:08 dumped_45mers.txt -rw-rw-r-- 1 pbrand pbrand 73484478 Mar 28 23:09 kmers_fq.fastq -rw-rw-r-- 1 pbrand pbrand 7814544 Mar 28 23:09 original_contigs_before_merging.fa -rw-rw-r-- 1 pbrand pbrand 13 Mar 28 21:45 reads_coverage.txt

[pbrec]

Also it produces two .gml files in the folder containing the read input files

Thanks!

[simoncchu]

@pbrec The initial assembly works, they are saved in "original_contigs_before_merging.fa" now. But the remove duplicate step fails. I think the TERefiner_1 doesn't runs well. Would you please check the responses under this issue https://github.com/Reedwarbler/REPdenovo/issues/4 ? It has a step show how to compile.