search

simoncchu / GAPPadder

GAPPadder is tool for closing gaps on draft genomes with short sequencing data
27 stars 7 forks source link

Assembly step not finishing #2

Open pganot opened 7 years ago

pganot commented 7 years ago

Hi back

I am opening a new issue, as gappadder isn't working yet with me: I have recompile TERefiner_1 and ContigsMerger (see previous closed issue). I am testing the program on a subset of denovo assembled scaffold (8 sequences with total of 6 gaps), as well as subset of miseq read pairs. Gappadder apparently runs fine but apparently halts in its final steps. Could you help me going through the whole program?

Here is the end of my log file:

Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa -c 0.99 -g ... Running command: ./ContigsMerger -s 0.4 -i1 -2.0 -i2 -2.0 -x 12 -y 50 -k 10 -t 5 -m 1 -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merge.info /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa > /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merged.fa ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merged.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merged.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -c 0.99 -g ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.no_contained.fa -c 0.99 ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa -c 0.99 -g ... Running command: ./ContigsMerger -s 0.4 -i1 -2.0 -i2 -2.0 -x 12 -y 50 -k 10 -t 5 -m 1 -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merge.info /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa > /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merged.fa ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merged.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merged.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -c 0.99 -g ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.no_contained.fa -c 0.99 ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa -c 0.99 -g ... Running command: ./ContigsMerger -s 0.4 -i1 -2.0 -i2 -2.0 -x 12 -y 50 -k 10 -t 5 -m 1 -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merge.info /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa > /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merged.fa ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merged.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merged.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -c 0.99 -g ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.no_contained.fa -c 0.99 ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa -c 0.99 -g ... Running command: ./ContigsMerger -s 0.4 -i1 -2.0 -i2 -2.0 -x 12 -y 50 -k 10 -t 5 -m 1 -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merge.info /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa > /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merged.fa ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merged.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merged.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -c 0.99 -g ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.no_contained.fa -c 0.99 ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa -c 0.99 -g ... Running command: ./ContigsMerger -s 0.4 -i1 -2.0 -i2 -2.0 -x 12 -y 50 -k 10 -t 5 -m 1 -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merge.info /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa > /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merged.fa ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merged.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merged.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -c 0.99 -g ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.no_contained.fa -c 0.99 ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa -c 0.99 -g ... Running command: ./ContigsMerger -s 0.4 -i1 -2.0 -i2 -2.0 -x 12 -y 50 -k 10 -t 5 -m 1 -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merge.info /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa > /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merged.fa ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merged.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merged.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -c 0.99 -g ... Running command: ./TERefiner_1 -P -b /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.itself.sort.bam -r /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa -o /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa_no_dup.fa.merged.fa.no_dup.fa.no_contained.fa -c 0.99 ... [bwa_index] Pack FASTA... [bwa_index] Pack FASTA... [bwa_index] Pack FASTA... [bwa_index] Pack FASTA... 0.00 sec [bwa_index] Construct BWT for the packed sequence... [bwa_index] 0.00 seconds elapse. [bwa_index] Update BWT... 0.00 sec [bwa_index] Pack forward-only FASTA... 0.00 sec [bwa_index] Construct SA from BWT and Occ... 0.00 sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa index /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa [main] Real time: 0.010 sec; CPU: 0.014 sec [M::bwa_idx_load_from_disk] read 0 ALT contigs [M::process] read 2 sequences (590 bp)... [M::mem_process_seqs] Processed 2 reads in 0.001 CPU sec, 0.001 real sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa mem -T 30 -a /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_1/contigs.fa /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/../flank_regions/1_1.fa [main] Real time: 0.004 sec; CPU: 0.007 sec 0.02 sec [bwa_index] Construct BWT for the packed sequence... 0.01 sec [bwa_index] Construct BWT for the packed sequence... 0.47 sec [bwa_index] Construct BWT for the packed sequence... [BWTIncCreate] textLength=53854680, availableWord=15789224 1.56 sec [bwa_index] Construct BWT for the packed sequence... [BWTIncCreate] textLength=166442510, availableWord=23711200 [bwa_index] 2.31 seconds elapse. [bwa_index] Update BWT... 0.01 sec [bwa_index] Pack forward-only FASTA... 0.02 sec [bwa_index] Construct SA from BWT and Occ... 2.47 sec [bwa_index] Construct BWT for the packed sequence... [BWTIncCreate] textLength=71479196, availableWord=17029476 0.20 sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa index /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa [main] Real time: 2.617 sec; CPU: 2.570 sec [M::bwa_idx_load_from_disk] read 0 ALT contigs [M::process] read 2 sequences (590 bp)... [M::mem_process_seqs] Processed 2 reads in 0.062 CPU sec, 0.061 real sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa mem -T 30 -a /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_2/contigs.fa /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/../flank_regions/2_2.fa [main] Real time: 0.074 sec; CPU: 0.071 sec [bwa_index] 3.83 seconds elapse. [bwa_index] Update BWT... 0.02 sec [bwa_index] Pack forward-only FASTA... 0.01 sec [bwa_index] Construct SA from BWT and Occ... 0.18 sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa index /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa [main] Real time: 4.307 sec; CPU: 4.062 sec [M::bwa_idx_load_from_disk] read 0 ALT contigs [M::process] read 2 sequences (590 bp)... [M::mem_process_seqs] Processed 2 reads in 0.072 CPU sec, 0.151 real sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa mem -T 30 -a /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_2/contigs.fa /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/../flank_regions/1_2.fa [main] Real time: 0.191 sec; CPU: 0.088 sec [BWTIncConstructFromPacked] 10 iterations done. 28090220 characters processed. [BWTIncConstructFromPacked] 10 iterations done. 26044312 characters processed. [BWTIncConstructFromPacked] 10 iterations done. 39112382 characters processed. [BWTIncConstructFromPacked] 20 iterations done. 48112776 characters processed. [bwa_index] 54.90 seconds elapse. [bwa_index] Update BWT... 0.41 sec [bwa_index] Pack forward-only FASTA... 0.31 sec [bwa_index] Construct SA from BWT and Occ... [BWTIncConstructFromPacked] 20 iterations done. 51892636 characters processed. 7.16 sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa index /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa [main] Real time: 96.273 sec; CPU: 63.264 sec [M::bwa_idx_load_from_disk] read 0 ALT contigs [M::process] read 2 sequences (590 bp)... [M::mem_process_seqs] Processed 2 reads in 0.053 CPU sec, 0.101 real sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa mem -T 30 -a /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/4_1/contigs.fa /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/../flank_regions/4_1.fa [main] Real time: 0.302 sec; CPU: 0.143 sec [BWTIncConstructFromPacked] 30 iterations done. 71479196 characters processed. [bwa_index] 63.42 seconds elapse. [bwa_index] Update BWT... 0.56 sec [bwa_index] Pack forward-only FASTA... 0.44 sec [bwa_index] Construct SA from BWT and Occ... 11.47 sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa index /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa [main] Real time: 117.312 sec; CPU: 78.371 sec [M::bwa_idx_load_from_disk] read 0 ALT contigs [M::process] read 2 sequences (590 bp)... [M::mem_process_seqs] Processed 2 reads in 0.089 CPU sec, 0.185 real sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa mem -T 30 -a /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/2_1/contigs.fa /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/../flank_regions/2_1.fa [main] Real time: 0.337 sec; CPU: 0.203 sec [BWTIncConstructFromPacked] 20 iterations done. 72256094 characters processed. [BWTIncConstructFromPacked] 30 iterations done. 101710254 characters processed. [BWTIncConstructFromPacked] 40 iterations done. 127885182 characters processed. [BWTIncConstructFromPacked] 50 iterations done. 151145470 characters processed. [bwa_index] 150.23 seconds elapse. [bwa_index] Update BWT... 1.37 sec [bwa_index] Pack forward-only FASTA... 0.95 sec [bwa_index] Construct SA from BWT and Occ... 27.63 sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa index /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa [main] Real time: 257.168 sec; CPU: 181.747 sec [M::bwa_idx_load_from_disk] read 0 ALT contigs [M::process] read 2 sequences (590 bp)... [M::mem_process_seqs] Processed 2 reads in 0.062 CPU sec, 0.137 real sec [main] Version: 0.7.15-r1140 [main] CMD: /media/physio/home/pganot/anaconda2/envs/genomeQC/bin/bwa mem -T 30 -a /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/velvet_temp/1_3/contigs.fa /media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/../flank_regions/1_3.fa [main] Real time: 0.567 sec; CPU: 0.305 sec Traceback (most recent call last): File "/media/physio/home/pganot/finisher/GAPPadder/main.py", line 283, in main_func(scommand,sfconfig) File "/media/physio/home/pganot/finisher/GAPPadder/main.py", line 274, in main_func gap_assembler.assemble_pipeline() File "/media/physio/home/pganot/finisher/GAPPadder/assemble_gaps.py", line 339, in assemble_pipeline id_remain=self.pick_already_constructed(contigs_select, fa_list, sf_picked) File "/media/physio/home/pganot/finisher/GAPPadder/assemble_gaps.py", line 321, in pick_already_constructed m_picked=contigs_select.get_already_picked(sf_picked) File "/media/physio/home/pganot/finisher/GAPPadder/pick_contigs.py", line 576, in get_already_picked with open(sf_picked) as fin_picked: IOError: [Errno 2] No such file or directory: u'/media/physio/home/pganot/finisher/GAPPadder/v143_sample/merged/../picked_seqs.fa'

Thanks Philippe