bowtie error

[hm22rice]

I get the following error:

+++ NOW PROCESSING Read alignment to reference genome [system] bowtie2-build -q ./data/reference.fasta ./02_reference_alignment/reference [system] bowtie2 -t -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x ./02_reference_alignment/reference -U ./01_sequence_conversion/P182_R1_001.converted.fastq -S ./02_reference_alignment/P182_R1_001.stage1.sam --un ./02_reference_alignment/P182_R1_001.stage1.unmatched.fastq Time loading reference: 00:00:00 Time loading forward index: 00:00:00 Time loading mirror index: 00:00:00 Multiseed full-index search: 00:00:21 933108 reads; of these: 933108 (100.00%) were unpaired; of these: 203144 (21.77%) aligned 0 times 711530 (76.25%) aligned exactly 1 time 18434 (1.98%) aligned >1 times 78.23% overall alignment rate Time searching: 00:00:21 Overall time: 00:00:21 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> FATAL ERROR <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Error running command: [system] bowtie2 -t -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x ./02_reference_alignment/reference -U ./01_sequence_conversion/P182_R1_001.converted.fastq -S ./02_reference_alignment/P182_R1_001.stage1.sam --un ./02_reference_alignment/P182_R1_001.stage1.unmatched.fastq Result code: 11 FILE: libbreseq/common.h LINE: 1445 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> STACK TRACE <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Backtrace with 6 stack frames. breseq() [0x43822a] breseq() [0x43afa5] breseq() [0x42849a] breseq() [0x40a1b8] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fa91e10c3d5] breseq() [0x417f17] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

breseq version 0.33.2 bowtie2 version 2.3.2

[jeffreybarrick]

Please try updating bowtie2 to the newest version. There are bugs in some previous versions.

I also recommend updating breseq for the same reason.

If those solutions don't work, please run the command that is failing:

bowtie2 -t -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x ./02_reference_alignment/reference -U ./01_sequence_conversion/P182_R1_001.converted.fastq -S ./02_reference_alignment/P182_R1_001.stage1.sam --un ./02_reference_alignment/P182_R1_001.stage1.unmatched.fastq

in your terminal and post the output.

[hm22rice]

Thank you for the quick response. I will update bowtie and breseq versions and run the data again.

Here's the output when I run the command that is failing: /projects/shamoo/hm22/bowtie2-2.3.2/bowtie2-align-s: /lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /projects/shamoo/hm22/bowtie2-2.3.2/bowtie2-align-s) /projects/shamoo/hm22/bowtie2-2.3.2/bowtie2-align-s: /lib64/libstdc++.so.6: versionGLIBCXX_3.4.20' not found (required by /projects/shamoo/hm22/bowtie2-2.3.2/bowtie2-align-s) (ERR): Description of arguments failed! Exiting now ...

[jeffreybarrick]

OK, so this indicates that there is an incompatibility between the version of bowtie2 and the platform you are running it on. It's definitely the bowtie2 install that is causing the problem.

[hm22rice]

Ok, updating bowtie 2 and breseq now. Thank you so much!

[hm22rice]

The latest versions of bowtie and breseq give me this error:

---> bowtie2 :: version 2.4.1 [/home/hm22/bowtie2-2.4.1/bowtie2] ---> R :: version 3.4.0 [/home/hm22/R/bin/R] +++ NOW PROCESSING Read and reference sequence file input READ FILE::EIP253_R1_001 Converting/filtering FASTQ file... Original base quality format: SANGER New format: SANGER Original reads: 1121306 bases: 168195900 Filtered reads: 4 bases: 600 (≥90.0% same base) Analyzed reads: 1121302 bases: 168195300 READ FILE::EIP253_R2_001 Converting/filtering FASTQ file... Original base quality format: SANGER New format: SANGER Original reads: 1121306 bases: 168195900 Filtered reads: 21 bases: 3150 (≥90.0% same base) Analyzed reads: 1121285 bases: 168192750 ::TOTAL:: Original reads: 2242612 bases: 336391800 Analyzed reads: 2242587 bases: 336388050 [samtools] faidx ./data/reference.fasta REFERENCE: CP009273 LENGTH: 4631469 +++ NOW PROCESSING Read alignment to reference genome [system] bowtie2-build -q "./data/reference.fasta" "./02_reference_alignment/reference" /usr/bin/env: python3: No such file or directory !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> FATAL ERROR <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Error running command: [system] bowtie2-build -q "./data/reference.fasta" "./02_reference_alignment/reference" Result code: 32512 FILE: libbreseq/common.h LINE: 1453 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> STACK TRACE <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Backtrace with 6 stack frames. breseq() [0x41d432] breseq() [0x43f1a1] breseq() [0x42ed07] breseq() [0x40b1fb] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fdd387c03d5] breseq() [0x41d0d7] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[jeffreybarrick]

Do you the same error when running the bowtie2 command as before? It seemed like you had a bowtie2 executable that was not compatible with your system. 32 vs 64 bit or Mac versus Linux, etc. What is your platform and how did you install bowtie2?

[hm22rice]

When I run this bowtie2 command: bowtie2-build -q "./data/reference.fasta" "./02_reference_alignment/reference" I get this message: /usr/bin/env: python3: No such file or directory

I am using Linux and I installed bowtie2 from here: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/2.4.1/ I built bowtie2 using the command "make NO_TBB=1"

[jeffreybarrick]

It looks like you need Python3 installed for bowtie2 to function. If you don't want to install python3, it looks like Version 2.3.5.1 is the last version that was compatible with Python2.

http://bowtie-bio.sourceforge.net/bowtie2/index.shtml

[hm22rice]

I'll try version 2.3.5.1 first and if that doesn't work, I'll talk to my system admin about installing python3. Thank you!

[hm22rice]

Unfortunately, with bowtie2 2.3.5.1, I get the same error

---> bowtie2 :: version 2.3.5.1 [/home/hm22/bowtie2-2.3.5.1/bowtie2] ---> R :: version 3.4.0 [/home/hm22/R/bin/R] --- ALREADY COMPLETE Read and reference sequence file input +++ NOW PROCESSING Read alignment to reference genome [system] bowtie2-build -q "./data/reference.fasta" "./02_reference_alignment/reference" [system] bowtie2 -t --no-unal -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x "./02_reference_alignment/reference" -U "./01_sequence_conversion/EIP253_R1_001.converted.fastq" -S "./02_reference_alignment/EIP253_R1_001.stage1.sam" --un "./02_reference_alignment/EIP253_R1_001.stage1.unmatched.fastq" Time loading reference: 00:00:00 Time loading forward index: 00:00:00 Time loading mirror index: 00:00:00 Multiseed full-index search: 00:00:25 1121302 reads; of these: 1121302 (100.00%) were unpaired; of these: 240812 (21.48%) aligned 0 times 860033 (76.70%) aligned exactly 1 time 20457 (1.82%) aligned >1 times 78.52% overall alignment rate Time searching: 00:00:25 Overall time: 00:00:25 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> FATAL ERROR <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Error running command: [system] bowtie2 -t --no-unal -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x "./02_reference_alignment/reference" -U "./01_sequence_conversion/EIP253_R1_001.converted.fastq" -S "./02_reference_alignment/EIP253_R1_001.stage1.sam" --un "./02_reference_alignment/EIP253_R1_001.stage1.unmatched.fastq" Result code: 11 FILE: libbreseq/common.h LINE: 1453 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> STACK TRACE <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Backtrace with 6 stack frames. breseq() [0x41d432] breseq() [0x43f1a1] breseq() [0x42f21d] breseq() [0x40b1fb] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7efd80f583d5] breseq() [0x41d0d7] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[jeffreybarrick]

What is the bowtie2 error this time if you run that? – These are all problems with your bowtie2 install.

[hm22rice]

[hm22@nlogin3 EIP253]$ bowtie2 -t --no-unal -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x "./02_reference_alignment/reference" -U "./01_sequence_conversion/EIP253_R1_001.converted.fastq" -S "./02_reference_alignment/EIP253_R1_001.stage1.sam" --un "./02_reference_alignment/EIP253_R1_001.stage1.unmatched.fastq" Time loading reference: 00:00:00 Time loading forward index: 00:00:00

Time loading mirror index: 00:00:00 Multiseed full-index search: 00:01:10 1121302 reads; of these: 1121302 (100.00%) were unpaired; of these: 240812 (21.48%) aligned 0 times 860033 (76.70%) aligned exactly 1 time 20457 (1.82%) aligned >1 times 78.52% overall alignment rate Time searching: 00:01:10 Overall time: 00:01:11 [hm22@nlogin3 EIP253]$

[jeffreybarrick]

Did you try deleting the entire breseq output directory and starting again?

[hm22rice]

I can do that now

[hm22rice]

I deleted the entire output directory and started again but got the same error. I also took a different fastq file in a fresh directory and tried to run the breseq command again but no luck

[jeffreybarrick]

That's really strange. I'm stumped. The bowtie2 command output you posted seems to indicate that step is working. But breseq thinks it is not working. The only thing I can think of is that maybe you are running a different bowtie2 copy on your system when you type the command versus when you run breseq? Maybe see if bowtie2 --version matches 2.3.5.1.

[hm22rice]

I think I am using the 2.3.5.1 version

[hm22@nlogin3 EIP252]$ bowtie2 --version /home/hm22/bowtie2-2.3.5.1/bowtie2-align-s version 2.3.5.1 64-bit Built on Wed Apr 17 02:38:57 UTC 2019 Compiler: gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) Options: -O3 -m64 -msse2 -funroll-loops -g3 -std=c++98 -DPOPCNT_CAPABILITY Sizeof {int, long, long long, void*, size_t, off_t}: {4, 8, 8, 8, 8, 8}

[jeffreybarrick]

Yes, It appears so. It seems to be giving a segmentation fault when breseq tries to run the same command. Did you also compile breseq to install it or did you download one of the binaries?

[hm22rice]

I compiled it

[jeffreybarrick]

What type of system are you on? What's the result of gcc --version?

You might try installing breseq using bioconda as an alternative. https://anaconda.org/bioconda/breseq

[hm22rice]

gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)

I am not familiar with bioconda but I will look into it. I will let you know how it goes. Thank you so much for your prompt responses and assistance.

[hm22rice]

I don't know if this matters but a different version of GCC gets loaded when breseq runs on my system:

The following have been reloaded with a version change: 1) GCC/4.8.3 => GCC/5.4.0

================================================================================ breseq 0.35.3 http://barricklab.org/breseq

Active Developers: Barrick JE, Deatherage DE Contact: jeffrey.e.barrick@gmail.com

breseq is free software; you can redistribute it and/or modify it under the terms the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version.

Copyright (c) 2008-2010 Michigan State University Copyright (c) 2011-2017 The University of Texas at Austin

If you use breseq in your research, please cite:

Deatherage, D.E., Barrick, J.E. (2014) Identification of mutations in laboratory-evolved microbes from next-generation sequencing data using breseq. Methods Mol. Biol. 1151: 165–188.

If you use structural variation (junction) predictions, please cite:

Barrick, J.E., Colburn, G., Deatherage D.E., Traverse, C.C., Strand, M.D., Borges, J.J., Knoester, D.B., Reba, A., Meyer, A.G. (2014) Identifying structural variation in haploid microbial genomes from short-read resequencing data using breseq. BMC Genomics 15:1039.

---> bowtie2 :: version 2.3.5.1 [/home/hm22/bowtie2-2.3.5.1/bowtie2] ---> R :: version 3.4.0 [/home/hm22/R/bin/R] +++ NOW PROCESSING Read and reference sequence file input READ FILE::EIP252_R1_001 Converting/filtering FASTQ file... Original base quality format: SANGER New format: SANGER Original reads: 1267184 bases: 190077600 Filtered reads: 2 bases: 300 (≥90.0% same base) Analyzed reads: 1267182 bases: 190077300 READ FILE::EIP252_R2_001 Converting/filtering FASTQ file... Original base quality format: SANGER New format: SANGER Original reads: 1267184 bases: 190077600 Filtered reads: 28 bases: 4200 (≥90.0% same base) Analyzed reads: 1267156 bases: 190073400 ::TOTAL:: Original reads: 2534368 bases: 380155200 Analyzed reads: 2534338 bases: 380150700 [samtools] faidx ./data/reference.fasta REFERENCE: CP009273 LENGTH: 4631469 +++ NOW PROCESSING Read alignment to reference genome [system] bowtie2-build -q "./data/reference.fasta" "./02_reference_alignment/reference" [system] bowtie2 -t --no-unal -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x "./02_reference_alignment/reference" -U "./01_sequence_conversion/EIP252_R1_001.converted.fastq" -S "./02_reference_alignment/EIP252_R1_001.stage1.sam" --un "./02_reference_alignment/EIP252_R1_001.stage1.unmatched.fastq" Time loading reference: 00:00:00 Time loading forward index: 00:00:00 Time loading mirror index: 00:00:00 Multiseed full-index search: 00:00:29 1267182 reads; of these: 1267182 (100.00%) were unpaired; of these: 267807 (21.13%) aligned 0 times 975676 (77.00%) aligned exactly 1 time 23699 (1.87%) aligned >1 times 78.87% overall alignment rate Time searching: 00:00:29 Overall time: 00:00:29 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> FATAL ERROR <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Error running command: [system] bowtie2 -t --no-unal -p 20 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x "./02_reference_alignment/reference" -U "./01_sequence_conversion/EIP252_R1_001.converted.fastq" -S "./02_reference_alignment/EIP252_R1_001.stage1.sam" --un "./02_reference_alignment/EIP252_R1_001.stage1.unmatched.fastq" Result code: 11 FILE: libbreseq/common.h LINE: 1453 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> STACK TRACE <!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Backtrace with 6 stack frames. breseq() [0x41d432] breseq() [0x43f1a1] breseq() [0x42f21d] breseq() [0x40b1fb] /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fb203a243d5] breseq() [0x41d0d7] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[hm22rice]

I was able to run breseq on conda.

Interestingly, today, when I tried to re-run some of the failed jobs from yesterday, they ran successfully (on the Linux version). I don't know what changed but after running 3 jobs successfully, I started getting the same error again. I am still very confused about this but for now, I will use breseq on bioconda.

Thank you!!

[jeffreybarrick]

Whew. Glad to hear that worked!

Too bad we can't figure it out, but I will close this for now.