samtools not found

[anusurendra]

Hi,

I have installed bwa-meth and was running the install version on the test examples on the Git page. When I try to create the reference index files I get the following error: Traceback (most recent call last): File "/usr/local/bin/bwameth.py", line 5, in <module> pkg_resources.run_script('bwameth==0.10', 'bwameth.py') File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 487, in run_script File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1337, in run_script File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 45, in <module> checkX('samtools') File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 43, in checkX raise Exception("executable for '%s' not found" % cmd)

I have samtools1.3.1 installed and included in the path and I am wondering how to fix the error?

Thanks in advance for your help.

[brentp]

it's not finding samtools on your path so you must have something set up wrong.

[anusurendra]

Hi Brent,

I appreciate your quick response. Here is the output from the echo $path command: /home/surendraa/bin:/home/surendraa/.local/bin:/home/surendraa/software/anaconda/4.1.1/bin:/home/surendraa/software/fastqc/0.115/FastQC:/home/surendraa/software/htslib/1.3.1/bin:/home/surendraa/software/samtools/1.3.1/bin:/home/surendraa/software/bfctools/1.3.1/bin:/home/surendraa/software/bowtie2/2.2.9:/home/surendraa/software/bwa/0.7.15:/home/surendraa/software/bismark/0.16.3:/home/surendraa/software/trimmomatic/0.36:/home/surendraa/software/blast/2.5.0/bin:/home/surendraa/software/MZmine/2.23:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/surendraa/software/fastqc/0.115/FastQC:/home/surendraa/software/anaconda/4.1.1/bin:/home/surendraa/software/htslib/1.3.1/bin:**/home/surendraa/software/samtools/1.3.1/bin**:/home/surendraa/software/bfctools/1.3.1/bin:/home/surendraa/software/bowtie2/2.2.9:/home/surendraa/software/bwa/0.7.15:/home/surendraa/software/bismark/0.16.3:/home/surendraa/software/trimmomatic/0.36:/snap/bin, Therefore, I am not sure why it is not finding samtools.

[brentp]

what happens when you type 'samtools' ? is it executable?

my samtools appears in the samtools directory, not in samtools/bin, perhaps that's your issue?

[anusurendra]

Brent,

I get the following when I type samtools:

`Program: samtools (Tools for alignments in the SAM format) Version: 1.3.1 (using htslib 1.3.1)

Usage: samtools [options]

Commands: -- Indexing dict create a sequence dictionary file faidx index/extract FASTA index index alignment

-- Editing calmd recalculate MD/NM tags and '=' bases fixmate fix mate information reheader replace BAM header rmdup remove PCR duplicates targetcut cut fosmid regions (for fosmid pool only) addreplacerg adds or replaces RG tags

-- File operations collate shuffle and group alignments by name cat concatenate BAMs merge merge sorted alignments mpileup multi-way pileup sort sort alignment file split splits a file by read group quickcheck quickly check if SAM/BAM/CRAM file appears intact fastq converts a BAM to a FASTQ fasta converts a BAM to a FASTA

-- Statistics bedcov read depth per BED region depth compute the depth flagstat simple stats idxstats BAM index stats phase phase heterozygotes stats generate stats (former bamcheck)

-- Viewing flags explain BAM flags tview text alignment viewer view SAM<->BAM<->CRAM conversion depad convert padded BAM to unpadded BAM`

[anusurendra]

Hi Brent,

I still have not resolved the samtools not found issue. However, when I run bwameth.py index ref.fa without sudo, I get the following error indexing: ref.fa.bwameth.c2t [bwa_index] Pack FASTA... [bns_fasta2bntseq] Failed to allocate 0 bytes at bntseq.c line 303: Success cmd was:bwa index -a bwtsw ref.fa.bwameth.c2t return code was:1 Traceback (most recent call last): File "/usr/local/bin/bwameth.py", line 5, in <module> pkg_resources.run_script('bwameth==0.10', 'bwameth.py') File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 487, in run_script File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1337, in run_script File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 601, in <module> main(sys.argv[1:]) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 548, in main sys.exit(bwa_index(convert_fasta(args[1]))) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 151, in bwa_index run("bwa index -a bwtsw %s" % fa) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 60, in run list(nopen("|%s" % cmd.lstrip("|"))) File "/usr/local/lib/python2.7/dist-packages/toolshed-0.4.6-py2.7.egg/toolshed/files.py", line 61, in process_iter raise ProcessException(cmd) toolshed.files.ProcessException: bwa index -a bwtsw ref.fa.bwameth.c2t

I am not sure if the 2 errors are related.

[brentp]

what is the output of ls -lh ref.fa.bwameth.c2t and if that's not empty (I assume it is), you could run:

bwa index -a bwtsw ref.fa.bwameth.c2t

if it is empty, delete it and retry.

[anusurendra]

Brent,

You are correct bwameth.c2t was empty. I deleted the file and ran bwameth.py index ref.fa and still get the same error again. Any further ideas as to how to resolve this issue?

[brentp]

what does ls -lh ref.fa show?

[anusurendra]

Brent,

The output shows -rw-rw-r-- 1 surendraa surendraa 3.6M Mar 9 13:34 ref.fa

[brentp]

so, starting with that and no other .c2t files what happens if you run:

rm -f ref.fa.bwameth*
python bwameth.py index ref.fa

[anusurendra]

Brent,

Here is the output Traceback (most recent call last): File "/usr/local/bin/bwameth.py", line 4, in <module> import pkg_resources File "/home/surendraa/software/anaconda/4.1.1/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/__init__.py", line 2927, in <module> File "/home/surendraa/software/anaconda/4.1.1/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/__init__.py", line 2913, in _call_aside File "/home/surendraa/software/anaconda/4.1.1/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/__init__.py", line 2940, in _initialize_master_working_set File "/home/surendraa/software/anaconda/4.1.1/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/__init__.py", line 635, in _build_master File "/home/surendraa/software/anaconda/4.1.1/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/__init__.py", line 943, in require File "/home/surendraa/software/anaconda/4.1.1/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/__init__.py", line 829, in resolve pkg_resources.DistributionNotFound: The 'bwameth==0.10' distribution was not found and is required by the application

[brentp]

you appear to have 2 different python's interacting. One of them is anaconda and the other is in /usr/local/lib/python2.7/dist-packages/

[anusurendra]

Brent,

I removed the anaconda version of python and also removed it from my path. So, now when I run which python I get /usr/bin/python. Then I ran /usr/bin/python2.7 /usr/local/bin/bwameth.py index ref.fa and now I get the following error: converting c2t in ref.fa to ref.fa.bwameth.c2t indexing: ref.fa.bwameth.c2t [bwa_index] Pack FASTA... [bns_fasta2bntseq] Failed to allocate 0 bytes at bntseq.c line 303: Success cmd was:bwa index -a bwtsw ref.fa.bwameth.c2t return code was:1 Traceback (most recent call last): File "/usr/local/bin/bwameth.py", line 5, in <module> pkg_resources.run_script('bwameth==0.10', 'bwameth.py') File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 487, in run_script File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1337, in run_script File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 601, in <module> main(sys.argv[1:]) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 548, in main sys.exit(bwa_index(convert_fasta(args[1]))) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 151, in bwa_index run("bwa index -a bwtsw %s" % fa) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 60, in run list(nopen("|%s" % cmd.lstrip("|"))) File "/usr/local/lib/python2.7/dist-packages/toolshed-0.4.6-py2.7.egg/toolshed/files.py", line 61, in process_iter raise ProcessException(cmd) toolshed.files.ProcessException: bwa index -a bwtsw ref.fa.bwameth.c2t. Not sure if now the best option is to reinstall both toolshed and bwameth?

[brentp]

when your command is failing, it's leaving an empty c2t file. You have to delete that each time. That's this error that you're showing.

[anusurendra]

Brent, I did delete the files when I ran the command and I get the above error.

[anusurendra]

Hi Brent,

I installed bwameth on a new system and am getting the same error: converting c2t in ref.fa to ref.fa.bwameth.c2t indexing: ref.fa.bwameth.c2t [bwa_index] Pack FASTA... [bns_fasta2bntseq] Failed to allocate 0 bytes at bntseq.c line 303: Success cmd was:bwa index -a bwtsw ref.fa.bwameth.c2t return code was:1 Traceback (most recent call last): File "/usr/local/bin/bwameth.py", line 4, in <module> __import__('pkg_resources').run_script('bwameth==0.10', 'bwameth.py') File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 719, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1504, in run_script exec(code, namespace, namespace) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 601, in <module> main(sys.argv[1:]) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 548, in main sys.exit(bwa_index(convert_fasta(args[1]))) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 151, in bwa_index run("bwa index -a bwtsw %s" % fa) File "/usr/local/lib/python2.7/dist-packages/bwameth-0.10-py2.7.egg/EGG-INFO/scripts/bwameth.py", line 60, in run list(nopen("|%s" % cmd.lstrip("|"))) File "/usr/local/lib/python2.7/dist-packages/toolshed/files.py", line 61, in process_iter raise ProcessException(cmd) toolshed.files.ProcessException: bwa index -a bwtsw ref.fa.bwameth.c2t.

I am really not sure where this issue is coming from.

[brentp]

I am not able to help you other than to suggest that there is something peculiar about the systems you are installing to as bwameth is fairly widely used without problems.

maybe you could update and re-build bwa.

[anusurendra]

Brent, I truly appreciate your patience and guidence. I was looking at the bwameth.py on the website and it is different than the script pulled from https://github.com/brentp/bwa-meth/archive/v0.10.tar.gz. Could this be an issue?

[brentp]

I doubt it. But you could try the latest from the repo.

[anusurendra]

Brent,

The issue was fixed after downloading and installing the new version.

[brentp]

ok. thanks for following up. glad you figured it out!