valueerror samtools GRCh37

[firatuyulur]

Hi, I installed bcbio and its working. I built my code as bcbio_nextgen.py -w template gatk-variant /home/firat/Documents/myfile1.csv /home/firat/Downloads/resources/exampleBAM.bam

This worked. and I was told;

Configuration file created at: /home/firat/myfile1/config/myfile1.yaml Edit to finalize and run with: cd /home/firat/myfile1/work bcbio_nextgen.py ../config/myfile1.yaml

And so i did;

firat@firat-X550CL:~$ cd /home/firat/myfile1/work firat@firat-X550CL:~/myfile1/work$ bcbio_nextgen.py ../config/myfile1.yaml

And here is what happened.

[2016-07-04T16:36Z] System YAML configuration: /usr/local/share/bcbio/galaxy/bcbio_system.yaml [2016-07-04T16:36Z] Resource requests: bwa, sambamba, samtools; memory: 2.00, 2.00; cores: 16, 16, 16 [2016-07-04T16:36Z] Configuring 1 jobs to run, using 1 cores each with 2.00g of memory reserved for each job [2016-07-04T16:36Z] Timing: organize samples [2016-07-04T16:36Z] multiprocessing: organize_samples [2016-07-04T16:36Z] Using input YAML configuration: /home/firat/myfile1/config/myfile1.yaml [2016-07-04T16:36Z] Checking sample YAML configuration: /home/firat/myfile1/config/myfile1.yaml [2016-07-04T16:36Z] Downloading GRCh37 samtools from AWS Traceback (most recent call last): File "/usr/local/bin/bcbio_nextgen.py", line 4, in import('pkg_resources').run_script('bcbio-nextgen==0.9.8', 'bcbio_nextgen.py') File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/init.py", line 726, in run_script

File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/init.py", line 1484, in run_script

File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-0.9.8-py2.7.egg-info/scripts/bcbio_nextgen.py", line 226, in main(kwargs) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio_nextgen-0.9.8-py2.7.egg-info/scripts/bcbio_nextgen.py", line 43, in main run_main(kwargs) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 43, in run_main fc_dir, run_info_yaml) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 87, in _run_toplevel for xs in pipeline(config, run_info_yaml, parallel, dirs, samples): File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 120, in variant2pipeline [x[0]["description"] for x in samples]]]) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel return run_multicore(fn, items, config, parallel=parallel) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore for data in joblib.Parallel(parallel["num_jobs"])(joblib.delayed(fn)(x) for x in items): File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 800, in call while self.dispatch_one_batch(iterator): File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 658, in dispatch_one_batch self._dispatch(tasks) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 566, in _dispatch job = ImmediateComputeBatch(batch) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 180, in init self.results = batch() File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in call return [func(_args, _kwargs) for func, args, kwargs in self.items] File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 51, in wrapper return apply(f, _args, _kwargs) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 287, in organize_samples return run_info.organize(*args) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 67, in organize item = add_reference_resources(item, remote_retriever) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 158, in add_reference_resources data["reference"] = genome.get_refs(data["genome_build"], aligner, data["dirs"]["galaxy"], data) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/genome.py", line 207, in get_refs galaxy_config, data) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/genome.py", line 159, in _get_ref_from_galaxy_loc cur_ref = download_prepped_genome(genome_build, data, name, need_remap) File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/genome.py", line 278, in download_prepped_genome raise ValueError("Could not find reference genome file %s %s" % (genome_build, name)) ValueError: Could not find reference genome file GRCh37 samtools

Whats the reason of having such error?

Thanks already

[chapmanb]

Thanks for trying out bcbio and sorry about the problem. How did you install bcbio? It looks like it's having a problem finding the reference data for GRCh37. Did you include this genome during your install?

http://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html#automated

Happy to provide more specific advice if it is present in your installation and it's not finding it for some reason. Hope this helps.

[firatuyulur]

Hi again, Thanks for the response. I installed it according to this guide: https://github.com/chapmanb/bcbio-nextgen-vm

Installation

Install bcbio-vm using conda with an isolated Miniconda Python and link to a location on your PATH:

wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh
bash Miniconda-latest-Linux-x86_64.sh -b -p ~/install/bcbio-vm/anaconda
~/install/bcbio-vm/anaconda/bin/conda install --yes -c bioconda bcbio-nextgen-vm
ln -s ~/install/bcbio-vm/anaconda/bin/bcbio_vm.py /usr/local/bin/bcbio_vm.py
ln -s ~/install/bcbio-vm/anaconda/bin/arvados-cwl-runner /usr/local/bin/arvados-cwl-runner
ln -s ~/install/bcbio-vm/anaconda/bin/cwltool /usr/local/bin/cwltool
ln -s ~/install/bcbio-vm/anaconda/bin/conda /usr/local/bin/bcbiovm_conda

would typing " --tooldir=/usr/local \ --genomes GRCh37 --aligners bwa --aligners bowtie2 " be enough to update bcbio to this level?

[chapmanb]

Thanks much for the details. You will need to install the biological data locally if you want to run analyses on your current. bcbio-vm is a bit more complex since it can also drive remote runs. In this case, you'd need to have Docker installed locally and also install biological data by following the rest of the installation instructions in the readme.

Practically, the bcbio-vm Docker setup is still under development so if you want the easiest install and run path your best bet is to use standard bcbio with the automated installer (and include the genomes and aligners you want as above):

http://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html#automated

Hope one of these paths gets things figured out for you.

[firatuyulur]

Hi again,

the reason why I used the alternative pathway for installation was because of having errors all the time. as I tried the http://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html#automated website for installation, here is the error I get;

$ python bcbio_nextgen_install.py /usr/local/share/bcbio --tooldir=/usr/local \

--genomes GRCh37 --aligners bwa --aligners bowtie2 Checking required dependencies Installing isolated base python installation Installing bcbio-nextgen --2016-07-05 22:30:06-- https://raw.githubusercontent.com/chapmanb/bcbio-nextgen/master/requirements-conda.txt Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 20 [text/plain] requirements-conda.txt: Permission denied

Cannot write to ‘requirements-conda.txt’ (Success). Traceback (most recent call last): File "bcbio_nextgen_install.py", line 245, in main(parser.parse_args(), sys.argv[1:]) File "bcbio_nextgen_install.py", line 38, in main bcbio = install_conda_pkgs(anaconda) File "bcbio_nextgen_install.py", line 70, in install_conda_pkgs subprocess.check_call(["wget", "--no-check-certificate", REMOTES["requirements"]]) File "/usr/lib/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['wget', '--no-check-certificate', 'https://raw.githubusercontent.com/chapmanb/bcbio-nextgen/master/requirements-conda.txt']' returned non-zero exit status 3

It recommends not to use sudo in the text. what should I do?

Thanks.

[roryk]

It looks like you don't have write permission to /usr/local/, so you can't install it there. If you can set tooldir to something you have write permission to you should be all set.