search

bcbio / bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis
https://bcbio-nextgen.readthedocs.io
MIT License
994 stars 354 forks source link

Installation Error: Samtools #944

Closed rb5000 closed 9 years ago

rb5000 commented 9 years ago

Hello,

I am trying to install an isolated version of BcBio on Ubuntu 14.04, due to no root access. However, when running the installer, I get this error:

[localhost] local: export CC=${CC:-which gcc} && export CXX=${CXX:-which g++} && export PERL5LIB=/home/robert/lib/perl5:${PERL5LIB} && /home/robert/bin/brew install --env=inherit  --without-curses samtools

Warning: local() encountered an error (return code 1) while executing 'export CC=${CC:-`which gcc`} && export CXX=${CXX:-`which g++`} && export PERL5LIB=/home/robert/lib/perl5:${PERL5LIB} && /home/robert/bin/brew install --env=inherit  --without-curses samtools'

Traceback (most recent call last):
  File "/home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/package/brew.py", line 263, in _install_pkg_latest
    cmd.replace("brew install", "brew install -v"))
ValueError: Failed to install brew formula: samtools
To debug, please try re-running the install command with verbose output:
export CC=${CC:-`which gcc`} && export CXX=${CXX:-`which g++`} && export PERL5LIB=/home/robert/lib/perl5:${PERL5LIB} && /home/robert/bin/brew install -v --env=inherit  --without-curses samtools
Traceback (most recent call last):
  File "bcbio_nextgen_install.py", line 275, in <module>
    main(parser.parse_args(), sys.argv[1:])
  File "bcbio_nextgen_install.py", line 46, in main
    subprocess.check_call([bcbio["bcbio_nextgen.py"], "upgrade"] + _clean_args(sys_argv, args, bcbio))
  File "/home/robert/anaconda/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/robert/anaconda/bin/bcbio_nextgen.py', 'upgrade', '--tooldir=/home/robert', '--genomes', 'hg19', '--aligners', 'bwa', '--aligners', 'bowtie2', '--data']' returned non-zero exit status 1

When trying to debug as recommended, and running the fix described in #745, I get this output:

==> Installing samtools dependency: htslib
==> Downloading https://github.com/samtools/htslib/archive/1.2.1.tar.gz
/home/robert/anaconda/bin/curl -fLA Homebrew 0.9.5 (Ruby 1.9.3-484; x86_64-linux) https://github.com/samtools/htslib/archive/1.2.1.tar.gz -C 0 -o /home/robert/.cache/Homebrew/htslib-1.2.1.tar.gz.incomplete
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (77) error setting certificate verify locations:
  CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
Error: Failed to download resource "htslib"
Download failed: https://github.com/samtools/htslib/archive/1.2.1.tar.gz

Any Suggestions?

Thanks, Robert

lpantano commented 9 years ago

Hi Robert,

sorry about that. This happens when curl form anaconda is used. See this related issue: #885

if you unlink curl from anaconda, or remove that path from PATH variable, should it work. bcbio doesn't need that path to be in you PATH. it would link the bcbio tools to the folder where you are installing the rest of the tools, but if you want the anaconda/bin there anyway you can try to unlink curl only.

hope this fixes the issue.

rb5000 commented 9 years ago

Thank you, I was able to unlink curl from anaconda, and I was able to get further in the installation process. However, now I have this error:

[localhost] local: wget --continue --no-check-certificate -O VarScan.v2.3.7.jar 'http://downloads.sourceforge.net/project/varscan/VarScan.v2.3.7.jar'
--2015-07-22 17:01:31--  http://downloads.sourceforge.net/project/varscan/VarScan.v2.3.7.jar
Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.34.181.59
Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.34.181.59|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2015-07-22 17:01:35 ERROR 404: Not Found.

Warning: local() encountered an error (return code 8) while executing 'wget --continue --no-check-certificate -O VarScan.v2.3.7.jar 'http://downloads.sourceforge.net/project/varscan/VarScan.v2.3.7.jar''

Traceback (most recent call last):
  File "/home/robert/Desktop/bcbio/anaconda/bin/bcbio_nextgen.py", line 207, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/home/robert/Desktop/bcbio/anaconda/lib/python2.7/site-packages/bcbio/install.py", line 81, in upgrade_bcbio
    upgrade_thirdparty_tools(args, REMOTES)
  File "/home/robert/Desktop/bcbio/anaconda/lib/python2.7/site-packages/bcbio/install.py", line 372, in upgrade_thirdparty_tools
    cbl_deploy.deploy(s)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 65, in deploy
    _setup_vm(options, vm_launcher, actions)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 110, in _setup_vm
    configure_instance(options, actions)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 255, in configure_instance
    install_biolinux(options)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 279, in install_biolinux
    _perform_install(target=target, flavor=flavor, more_custom_add=options.get("custom_add", None))
  File "/home/robert/tmpbcbio-install/cloudbiolinux/fabfile.py", line 112, in _perform_install
    _custom_installs(pkg_install, custom_ignore, custom_add)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/fabfile.py", line 176, in _custom_installs
    install_custom(p, True, pkg_to_group)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/fabfile.py", line 280, in install_custom
    fn(env)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/custom/bio_nextgen.py", line 454, in install_varscan
    out_file = shared._remote_fetch(env, url)
  File "/home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/custom/shared.py", line 201, in _remote_fetch
    raise IOError("Failure to retrieve remote file")
IOError: Failure to retrieve remote file
Traceback (most recent call last):
  File "bcbio_nextgen_install.py", line 275, in <module>
    main(parser.parse_args(), sys.argv[1:])
  File "bcbio_nextgen_install.py", line 46, in main
    subprocess.check_call([bcbio["bcbio_nextgen.py"], "upgrade"] + _clean_args(sys_argv, args, bcbio))
  File "/home/robert/anaconda/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/robert/Desktop/bcbio/anaconda/bin/bcbio_nextgen.py', 'upgrade', '--tooldir=/home/robert/Desktop/bcbio', '--isolate', '--genomes', 'hg19', '--aligners', 'bwa', '--aligners', 'bowtie2', '--data']' returned non-zero exit status 1

Thanks, Robert

lpantano commented 9 years ago

Hi,

sorry about that. Did you try the command to know if it works outside the installation script?

I would give it another shot. sometimes sourceforge is down a while but then comes back.

if not let us know if the command to download varscan works if you run it alone.

ohofmann commented 9 years ago

Yes, that happened to me during a recent re-install, too. Restarting five minutes later and it completed without problems. SourceForge is entirely unreliable by now; I was hoping that they'd move VarScan to BitBucket or GitHub one of these days...

rb5000 commented 9 years ago

Thank you for the advice, I retried the installation and it was able to finish :)

However, when attempting to upgrade tools and install GATK, I now get this error:

INFO: Reading /home/robert/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/packages-homebrew.yaml
DBG [shared.py]: Packages to install: bcftools,cufflinks-binary,htslib,samtools-library-0.1,samtools;--without-curses,bamtools,bedtools,biobambam,cramtools,express-binary,fastqc,grabix,kraken,oncofuse,picard-tools,qualimap,sambamba-binary,samblaster,seqtk,sickle,speedseq,staden_io_lib,bcbio-variation-recall,cancerit-allelecount,delly;--with-binary,freebayes,gatk-framework,hall-lab-sv-tools,impute2,lofreq,lumpy-sv,pindel,platypus-variant,qsignature,rtg-tools,scalpel,snpeff,theta2,vardict,vardict-java,vcflib,vep,vt;--with-binary,wham;--with-binary,bowtie2,bwa,bwakit,novoalign,rna-star,tophat-binary,cmake,p7zip,pkg-config,s3gof3r,cpanminus
[localhost] local: /home/robert/Desktop/bcbio/bin/brew tap
[localhost] local: /home/robert/Desktop/bcbio/bin/brew --prefix
[localhost] local: git config user.email 'stash@brew.sh'
[localhost] local: git stash --quiet
.gitignore: needs merge
.gitignore: needs merge
.gitignore: unmerged (59abbf727aa58fbedfe9adf5ff8638499103068d)
.gitignore: unmerged (6c3b7d35bed26f78da037b144bf0f1e70ea0da87)
fatal: git-write-tree: error building trees
Cannot save the current index state

Fatal error: local() encountered an error (return code 1) while executing 'git stash --quiet'

Aborting.

I tried deleting the tmpbcbio-install and running the command again, but it still provided the same error.

Any suggestions?

Thank you, Robert

chapmanb commented 9 years ago

Robert; Glad a restart got everything further along and apologies about the issues. There have been some changes recently to homebrew, which we use to install packages, that can cause issues on some systems. I pushed some fixes which I hope will resolve your problem. If you get rid of the tmpbcbio-install directory again and re-run, it should pull in the fixed version and work cleanly for you. Hope this gets things running for you.

rb5000 commented 9 years ago

I deleted the directories and was able to run the command successfully. Thank you for all your help!