search

bcbio / bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis
https://bcbio-nextgen.readthedocs.io
MIT License
992 stars 354 forks source link

Both install script and docker-files fails to build #510

Closed thashim closed 10 years ago

thashim commented 10 years ago

Both dockerfile build and the install script run on a clean ubuntu docker vm + dependencies fail with the same error of: no attribute short_name.

Log below of docker build with run string "docker build . > log.txt"


Step 0 : FROM stackbrew/ubuntu:14.04
 ---> e54ca5efa2e9
Step 1 : MAINTAINER Brad Chapman "https://github.com/chapmanb"
 ---> Using cache
 ---> c5d53712d1e7
Step 2 : RUN apt-get update && apt-get install -y build-essential zlib1g-dev wget curl python-setuptools git
 ---> Using cache
 ---> c97877fb71f5
Step 3 : RUN mkdir -p /tmp/fuse-hack && cd /tmp/fuse-hack &&    apt-get install libfuse2 &&    apt-get download fuse &&    dpkg-deb -x fuse_* . &&    dpkg-deb -e fuse_* &&    rm fuse_*.deb &&    echo -en '#!/bin/bash\nexit 0\n' > DEBIAN/postinst &&    dpkg-deb -b . /fuse.deb &&    dpkg -i /fuse.deb &&    rm -rf /tmp/fuse-hack
 ---> Using cache
 ---> 0cf108dc9a46
Step 4 : RUN git config --global url.https://github.com/.insteadOf git://github.com/ &&    mkdir -p /tmp/bcbio-nextgen-install && cd /tmp/bcbio-nextgen-install &&    wget --no-check-certificate      https://raw.github.com/chapmanb/bcbio-nextgen/master/scripts/bcbio_nextgen_install.py &&    python bcbio_nextgen_install.py /usr/local/share/bcbio-nextgen      --nosudo --nodata -u development
 ---> Using cache
 ---> 8655fb50450d
Step 5 : RUN /usr/local/share/bcbio-nextgen/anaconda/bin/bcbio_nextgen.py upgrade --tooldir=/usr/local --tools
 ---> Running in 89c943e8d1d2
Cloning into 'cloudbiolinux'...
INFO: <cloudbio.flavor.Flavor instance at 0x7fbbdd1d9a70>
INFO: This is a ngs_pipeline_minimal flavor
INFO: Reading default fabricrc.txt
DBG [config.py]: Using config file /tmpbcbio-install/cloudbiolinux/cloudbio/../config/fabricrc.txt
INFO: Distribution __auto__
INFO: Get local environment
INFO: Ubuntu setup
DBG [distribution.py]: Debian-shared setup
DBG [distribution.py]: Source=trusty
DBG [distribution.py]: Checking target distribution ubuntu
DBG [distribution.py]: NixPkgs: Ignored
INFO: Now, testing connection to host...
INFO: Connection to host appears to work!
DBG [utils.py]: Expand paths
DBG [config.py]: Using config file /tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
INFO: Meta-package information from /tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
- Packages: minimal,libraries,perl,python,java,r,ruby,bio_nextgen
- Libraries: r-libs
/
DBG [deb.py]: _setup_apt_sources /etc/apt/sources.list.d/cloudbiolinux.list
DBG [deb.py]: Source deb http://us.archive.ubuntu.com/ubuntu/ trusty universe
DBG [deb.py]: Source deb http://us.archive.ubuntu.com/ubuntu/ trusty multiverse
DBG [deb.py]: Source deb http://us.archive.ubuntu.com/ubuntu/ trusty-updates universe
DBG [deb.py]: Source deb http://us.archive.ubuntu.com/ubuntu/ trusty-updates multiverse
DBG [deb.py]: Source deb http://archive.canonical.com/ubuntu trusty partner
DBG [deb.py]: Source deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen
DBG [deb.py]: Source deb http://cran.fhcrc.org/bin/linux/ubuntu trusty/
DBG [deb.py]: Source deb http://archive.cloudera.com/debian maverick-cdh3 contrib
DBG [deb.py]: Source deb http://archive.canonical.com/ubuntu trusty partner
DBG [deb.py]: Source deb http://ppa.launchpad.net/freenx-team/ppa/ubuntu precise main
DBG [deb.py]: Source deb http://ppa.launchpad.net/nebc/bio-linux/ubuntu precise main
DBG [deb.py]: Source deb [arch=amd64 trusted=yes] http://research.cs.wisc.edu/htcondor/debian/stable/ squeeze contrib
DBG [deb.py]: Source deb http://nebc.nerc.ac.uk/bio-linux/ unstable bio-linux
DBG [deb.py]: Source deb http://download.virtualbox.org/virtualbox/debian trusty contrib
Upgrading third party tools to latest versions
Setting up virtual machine
[localhost] local: cat /etc/*release | grep DISTRIB_CODENAME | cut -f 2 -d =
[localhost] local: cat /proc/version
[localhost] local: cat /etc/*release | grep DISTRIB_CODENAME | cut -f 2 -d =
[localhost] local: echo $HOME
[localhost] local: uname -m
[localhost] local: whoami
[localhost] local: echo $HOME
[localhost] local: touch /etc/apt/sources.list.d/cloudbiolinux.list
[localhost] local: echo '# This file was modified for ngs_pipeline_minimal' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://us.archive.ubuntu.com/ubuntu/ trusty universe' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://us.archive.ubuntu.com/ubuntu/ trusty multiverse' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://us.archive.ubuntu.com/ubuntu/ trusty-updates universe' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://us.archive.ubuntu.com/ubuntu/ trusty-updates multiverse' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://archive.canonical.com/ubuntu trusty partner' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://cran.fhcrc.org/bin/linux/ubuntu trusty/' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://archive.cloudera.com/debian maverick-cdh3 contrib' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://ppa.launchpad.net/freenx-team/ppa/ubuntu precise main' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://ppa.launchpad.net/nebc/bio-linux/ubuntu precise main' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb [arch=amd64 trusted=yes] http://research.cs.wisc.edu/htcondor/debian/stable/ squeeze contrib' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://nebc.nerc.ac.uk/bio-linux/ unstable bio-linux' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'deb http://download.virtualbox.org/virtualbox/debian trusty contrib' >> "$(echo /etc/apt/sources.list.d/cloudbiolinux.list)"
[localhost] local: echo 'export DEBIAN_FRONTEND=noninteractive' >> "$(echo ~/.bashrc)"
[localhost] local: sed -i.bak -r -e 's/^([ ]+\*\) return;;)$/#\1/g' "$(echo ~/.bashrc)"
[localhost] local: echo "postfix postfix/not_configured boolean true" | /usr/bin/debconf-set-selections;echo "postfix postfix/main_mailer_type select 'No configuration'" | /usr/bin/debconf-set-selections;echo "mysql-server-5.1 mysql-server/root_password string '(password omitted)'" | /usr/bin/debconf-set-selections;echo "mysql-server-5.1 mysql-server/root_password_again string '(password omitted)'" | /usr/bin/debconf-set-selections;echo "mysql-server-5.5 mysql-server/root_password string '(password omitted)'" | /usr/bin/debconf-set-selections;echo "mysql-server-5.5 mysql-server/root_password_again string '(password omitted)'" | /usr/bin/debconf-set-selections;echo "sun-java6-jdk shared/accepted-sun-dlj-v1-1 select true" | /usr/bin/debconf-set-selections;echo "sun-java6-jre shared/accepted-sun-dlj-v1-1 select true" | /usr/bin/debconf-set-selections;echo "sun-java6-bin shared/accepted-sun-dlj-v1-1 select true" | /usr/bin/debconf-set-selections;echo "grub-pc grub2/linux_cmdline string ''" | /usr/bin/debconf-set-selections;echo "grub-pc grub-pc/install_devices_empty boolean true" | /usr/bin/debconf-set-selections;echo "acroread acroread/default-viewer boolean false" | /usr/bin/debconf-set-selections;echo "rabbitmq-server rabbitmq-server/upgrade_previous note" | /usr/bin/debconf-set-selections;echo "condor condor/wantdebconf boolean false" | /usr/bin/debconf-set-selections;echo "ttf-mscorefonts-installer msttcorefonts/acINFO: Update GPG keys for repositories
OK
OK
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.cdWz7heo4U --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv 7F0CEB10
gpg: requesting key 7F0CEB10 from hkp server keyserver.ubuntu.com
gpg: key 7F0CEB10: public key "Richard Kreuter <richard@10gen.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.sWfJqy3cS6 --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv E084DAB9
gpg: requesting key E084DAB9 from hkp server keyserver.ubuntu.com
gpg: key E084DAB9: public key "Michael Rutter <marutter@gmail.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.EDNCsV2XZT --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver subkeys.pgp.net --recv D018A4CE
gpg: requesting key D018A4CE from hkp server subkeys.pgp.net
gpg: keyserver timed out
gpg: keyserver receive failed: keyserver error

Warning: local() encountered an error (return code 2) while executing 'apt-key adv --keyserver subkeys.pgp.net --recv D018A4CE'

Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.kNQa1EiHqF --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv D67FC6EAE2A11821
gpg: requesting key E2A11821 from hkp server keyserver.ubuntu.com
gpg: key E2A11821: public key "Vincent Goulet <vincent.goulet@act.ulaval.ca>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Ign http://downloads-distro.mongodb.org dist InRelease
Ign http://us.archive.ubuntu.com trusty InRelease
Ign http://archive.ubuntu.com trusty InRelease
Ign http://archive.ubuntu.com trusty-updates InRelease
Ign http://us.archive.ubuntu.com trusty-updates InRelease
Ign http://archive.canonical.com trusty InRelease
Ign http://archive.cloudera.com maverick-cdh3 InRelease
Ign http://ppa.launchpad.net precise InRelease
Ign http://ppa.launchpad.net precise InRelease
Ign http://nebc.nerc.ac.uk unstable InRelease
Get:1 http://download.virtualbox.org trusty InRelease [5628 B]
Ign http://archive.ubuntu.com trusty-security InRelease
Get:2 http://downloads-distro.mongodb.org dist Release.gpg [490 B]
Ign http://research.cs.wisc.edu squeeze InRelease
Hit http://archive.ubuntu.com trusty Release.gpg
Get:3 http://us.archive.ubuntu.com trusty Release.gpg [933 B]
Get:4 http://us.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Get:5 http://archive.ubuntu.com trusty-updates Release.gpg [933 B]
Get:6 http://us.archive.ubuntu.com trusty Release [58.5 kB]
Get:7 http://archive.ubuntu.com trusty-security Release.gpg [933 B]
Get:8 http://downloads-distro.mongodb.org dist Release [2032 B]
Hit http://archive.ubuntu.com trusty Release
Get:9 http://archive.cloudera.com maverick-cdh3 Release.gpg [198 B]
Ign http://cran.fhcrc.org trusty/ InRelease
Get:10 http://archive.ubuntu.com trusty-updates Release [58.5 kB]
Ign http://research.cs.wisc.edu squeeze Release.gpg
Get:11 http://us.archive.ubuntu.com trusty-updates Release [58.5 kB]
Get:12 http://archive.cloudera.com maverick-cdh3 Release [2428 B]
Get:13 http://archive.ubuntu.com trusty-security Release [58.5 kB]
Get:14 http://ppa.launchpad.net precise Release.gpg [316 B]
Get:15 http://archive.canonical.com trusty Release.gpg [933 B]
Ign http://research.cs.wisc.edu squeeze Release
Get:16 http://nebc.nerc.ac.uk unstable Release.gpg [198 B]
Get:17 http://cran.fhcrc.org trusty/ Release.gpg [490 B]
Hit http://archive.ubuntu.com trusty/main Sources
Hit http://archive.ubuntu.com trusty/restricted Sources
Get:18 http://downloads-distro.mongodb.org dist/10gen amd64 Packages [19.0 kB]
Get:19 http://us.archive.ubuntu.com trusty/universe amd64 Packages [5859 kB]
Hit http://archive.ubuntu.com trusty/universe Sources
Hit http://archive.ubuntu.com trusty/main amd64 Packages
Get:20 http://download.virtualbox.org trusty/contrib amd64 Packages [997 B]
Hit http://archive.ubuntu.com trusty/restricted amd64 Packages
Hit http://archive.ubuntu.com trusty/universe amd64 Packages
Get:21 http://archive.ubuntu.com trusty-updates/main Sources [83.9 kB]
Get:22 http://archive.ubuntu.com trusty-updates/restricted Sources [14 B]
Get:23 http://archive.canonical.com trusty Release [8257 B]
Get:24 http://archive.cloudera.com maverick-cdh3/contrib amd64 Packages [14.0 kB]
Get:25 http://ppa.launchpad.net precise Release.gpg [316 B]
Get:26 http://nebc.nerc.ac.uk unstable Release [2424 B]
Get:27 http://archive.ubuntu.com trusty-updates/universe Sources [60.0 kB]
Get:28 http://cran.fhcrc.org trusty/ Release [2272 B]
Get:29 http://archive.ubuntu.com trusty-updates/main amd64 Packages [218 kB]
Ign http://nebc.nerc.ac.uk unstable Release
Get:30 http://archive.ubuntu.com trusty-updates/restricted amd64 Packages [14 B]
Get:31 http://archive.ubuntu.com trusty-updates/universe amd64 Packages [155 kB]
Get:32 http://research.cs.wisc.edu squeeze/contrib amd64 Packages [2648 B]
Get:33 http://archive.ubuntu.com trusty-security/main Sources [32.1 kB]
Get:34 http://ppa.launchpad.net precise Release [11.9 kB]
Ign http://ppa.launchpad.net precise Release
Get:35 http://archive.canonical.com trusty/partner amd64 Packages [4917 B]
Get:36 http://cran.fhcrc.org trusty/ Packages [16.0 kB]
Err http://nebc.nerc.ac.uk unstable/bio-linux amd64 Packages

Get:37 http://archive.ubuntu.com trusty-security/restricted Sources [14 B]
Get:38 http://archive.ubuntu.com trusty-security/universe Sources [6822 B]
Get:39 http://archive.ubuntu.com trusty-security/main amd64 Packages [107 kB]
Get:40 http://archive.ubuntu.com trusty-security/restricted amd64 Packages [14 B]
Get:41 http://archive.ubuntu.com trusty-security/universe amd64 Packages [37.2 kB]
Get:42 http://ppa.launchpad.net precise Release [11.9 kB]
Ign http://ppa.launchpad.net precise Release
Err http://nebc.nerc.ac.uk unstable/bio-linux amd64 Packages

Get:43 http://us.archive.ubuntu.com trusty/multiverse amd64 Packages [132 kB]
Get:44 http://ppa.launchpad.net precise/main amd64 Packages [5009 B]
Get:45 http://us.archive.ubuntu.com trusty-updates/universe amd64 Packages [155 kB]
Get:46 http://us.archive.ubuntu.com trusty-updates/multiverse amd64 Packages [7392 B]
Err http://nebc.nerc.ac.uk unstable/bio-linux amd64 Packages

Get:47 http://ppa.launchpad.net precise/main amd64 Packages [71.8 kB]
Get:48 http://nebc.nerc.ac.uk unstable/bio-linux amd64 Packages [46.0 kB]
Fetched 7322 kB in 5s (1299 kB/s)
Reading package lists...
W: GPG error: http://nebc.nerc.ac.uk unstable Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B1A598E8128B92BD
W: GPG error: http://ppa.launchpad.net precise Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 2A8E3034D018A4CE
W: GPG error: http://ppa.launchpad.net precise Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 759A8E0E5BA15160
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  bio-linux-keyring
0 upgraded, 1 newly installed, 0 to remove and 31 not upgraded.
Need to get 8990 B of archives.
After this operation, 0 B of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  bio-linux-keyring
Get:1 http://nebc.nerc.ac.uk/bio-linux/ unstable/bio-linux bio-linux-keyring all 6 [8990 B]
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin: 
Fetched 8990 B in 0s (48.3 kB/s)
Selecting previously unselected package bio-linux-keyring.
(Reading database ... 19236 files and directories currently installed.)
Preparing to unpack .../bio-linux-keyring_6_all.deb ...
Unpacking bio-linux-keyring (6) ...
Setting up bio-linux-keyring (6) ...
OK
OK
OK
OK
OK
OK
cepted-mscorefonts-eula boolean true" | /usr/bin/debconf-set-selections;echo "ttf-mscorefonts-installer msttcorefonts/present-mscorefonts-eula note" | /usr/bin/debconf-set-selections;echo "gdm shared/default-x-display-manager select gdm" | /usr/bin/debconf-set-selections;echo "lightdm shared/default-x-display-manager select gdm" | /usr/bin/debconf-set-selections;echo "postfix postfix/mailname string notusedexample.org" | /usr/bin/debconf-set-selections;echo "debconf debconf/priority select critical" | /usr/bin/debconf-set-selections;
[localhost] local: wget -q -O- http://archive.cloudera.com/debian/archive.key | apt-key add -
[localhost] local: wget -q -O- http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc | apt-key add -
[localhost] local: apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
[localhost] local: apt-key adv --keyserver keyserver.ubuntu.com --recv E084DAB9
[localhost] local: apt-key adv --keyserver subkeys.pgp.net --recv D018A4CE
[localhost] local: apt-key adv --keyserver keyserver.ubuntu.com --recv D67FC6EAE2A11821
[localhost] local: apt-get update
[localhost] local: sudo apt-get install -y --force-yes bio-linux-keyring
Traceback (most recent call last):
  File "/usr/local/share/bcbio-nextgen/anaconda/bin/bcbio_nextgen.py", line 199, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/install.py", line 62, in upgrade_bcbio
    upgrade_thirdparty_tools(args, REMOTES)
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/install.py", line 288, in upgrade_thirdparty_tools
    cbl_deploy.deploy(s)
  File "/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 65, in deploy
    _setup_vm(options, vm_launcher, actions)
  File "/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 110, in _setup_vm
    configure_instance(options, actions)
  File "/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 255, in configure_instance
    install_biolinux(options)
  File "/tmpbcbio-install/cloudbiolinux/cloudbio/deploy/__init__.py", line 279, in install_biolinux
    _perform_install(target=target, flavor=flavor, more_custom_add=options.get("custom_add", None))
  File "/tmpbcbio-install/cloudbiolinux/fabfile.py", line 105, in _perform_install
    _configure_and_install_native_packages(env, pkg_install)
  File "/tmpbcbio-install/cloudbiolinux/cloudbio/package/__init__.py", line 28, in _configure_and_install_native_packages
    _apt_packages(pkg_install)
  File "/tmpbcbio-install/cloudbiolinux/cloudbio/package/deb.py", line 25, in _apt_packages
    if env.flavor.short_name not in ["minimal"]:
AttributeError: Flavor instance has no attribute 'short_name'

chapmanb commented 10 years ago

Thanks much for the report and sorry for the issue. We're re-working some of the installer internals today and this was a bug. I pushed a fix for this that should let you proceed. For local installs, you'll want to remove the temporary directory (rm -rf tmpbcbio-install) to get a fresh CloudBioLinux cloned for installation. Please let us know if you run into any more issues. I'll step through a clean install as well and check for other problems. Thanks again.