bcbio / bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis
https://bcbio-nextgen.readthedocs.io
MIT License
991 stars 354 forks source link

1.1.5 fails to install deepvariant when upgrading tools #2789

Closed naumenko-sa closed 5 years ago

naumenko-sa commented 5 years ago

Hello!

A fresh install of bcbio-nextgen 1.1.5 with --nodata and without --tools went well (I removed the environments from old installations in ~/.conda/environments.txt, cleaned PATH and PYTHONPATH from old bcbio or python entries).

Then (PYTHONPATH, PATH, ~/.conda/environments.txt pointed to the new installation) bcbio_nextgen.py upgrade -u skip --tools --tooldir [tooldir] failed with

Executing transaction: ...working... ERROR conda.core.link:_execute_post_link_actions(658): An error occurred while installing package 'bioconda::deepvariant-0.8.0-py27h7333d49_0'.
LinkError: post-link script failed for package bioconda::deepvariant-0.8.0-py27h7333d49_0
running your command again with `-v` will provide additional information
location of failed script: /hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/envs/dv/bin/.deepvariant-post-link.sh
==> script messages <==
<None>

Attempting to roll back.

failed
ERROR conda.core.link:_execute(568): An error occurred while installing package 'bioconda::deepvariant-0.8.0-py27h7333d49_0'.
LinkError: post-link script failed for package bioconda::deepvariant-0.8.0-py27h7333d49_0
running your command again with `-v` will provide additional information
location of failed script: /hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/envs/dv/bin/.deepvariant-post-link.sh
==> script messages <==
<None>

Attempting to roll back.

Rolling back transaction: ...working... done

LinkError: post-link script failed for package bioconda::deepvariant-0.8.0-py27h7333d49_0
running your command again with `-v` will provide additional information
location of failed script: /hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/envs/dv/bin/.deepvariant-post-link.sh
==> script messages <==
<None>

/hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/tmpbcbio-install/cloudbiolinux/cloudbio/package/shared.py:11: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  full_data = yaml.load(in_handle)
Upgrading bcbio
Upgrading third party tools to latest versions
Reading packages from /hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/tmpbcbio-install/cloudbiolinux/contrib/flavor/ngs_pipeline_minimal/packages-conda.yaml
Checking for problematic or migrated packages in default environment
# Installing into conda environment default: age-metasv, bamtools=2.4.0, bamutil, bbmap, bcbio-prioritize, bcbio-variation, bcbio-variation-recall, bcftools, bedtools=2.27.1, bio-vcf, biobambam, bowtie, bowtie2, break-point-inspector, bwa, bwakit, cage, cancerit-allelecount, chipseq-greylist, cnvkit, coincbc, cramtools, cufflinks, cyvcf2, deeptools, delly, duphold>=0.0.9, ensembl-vep=95.*, express, extract-sv-reads, fastp, fastqc>=0.11.8=1, fgbio, freebayes=1.1.0.46, gatk, gatk4, geneimpacts, genesplicer, gffcompare, goleft, grabix, gridss, gsort, gvcfgenotyper, h5py, hmftools-amber, hmftools-cobalt, hmftools-purple, hmmlearn, hts-nim-tools, htslib, impute2, kallisto>=0.43.1, kraken, lofreq, matplotlib<3.0.0,>=2.1.1, maxentscan, mbuffer, minimap2, mintmap, mirdeep2=2.0.0.7, mirtop, moreutils, mosdepth, multiqc, multiqc-bcbio, ngs-disambiguate, novoalign, octopus>=0.5.1b, oncofuse, parallel, pbgzip, perl-sanger-cgp-battenberg, picard, pindel, pizzly, preseq, pyloh, pysam>=0.14.0, pythonpy, qsignature, qualimap, rapmap, razers3=3.5.0, rtg-tools, sailfish, salmon, sambamba, samblaster, samtools, scalpel, seq2c, seqbuster, seqcluster, seqtk, sickle-trim, simple_sv_annotation, singlecell-barcodes, snap-aligner=1.0dev.97, snpeff=4.3.1t, solvebio, spades, star=2.6.1d, stringtie, subread, survivor, tdrmapper, tophat-recondition, ucsc-bedgraphtobigwig, ucsc-bedtobigbed, ucsc-bigbedinfo, ucsc-bigbedsummary, ucsc-bigbedtobed, ucsc-bigwiginfo, ucsc-bigwigsummary, ucsc-bigwigtobedgraph, ucsc-bigwigtowig, ucsc-fatotwobit, ucsc-gtftogenepred, ucsc-liftover, ucsc-wigtobigwig, vardict, vardict-java, variantbam, varscan, vcfanno, vcflib, verifybamid2, viennarna, vqsr_cnn, vt, wham, anaconda-client, awscli, bzip2, ncurses, nodejs, p7zip, readline, s3gof3r, xz, perl-app-cpanminus, perl-archive-extract, perl-archive-zip, perl-bio-db-sam, perl-cgi, perl-dbi, perl-encode-locale, perl-file-fetch, perl-file-sharedir, perl-file-sharedir-install, perl-ipc-system-simple, perl-lwp-protocol-https, perl-lwp-simple, perl-statistics-descriptive, perl-time-hires, perl-vcftools-vcf, bioconductor-annotate, bioconductor-biocgenerics, bioconductor-biocinstaller, bioconductor-biocstyle, bioconductor-biostrings, bioconductor-biovizbase, bioconductor-bsgenome.hsapiens.ucsc.hg19, bioconductor-bsgenome.hsapiens.ucsc.hg38, bioconductor-bubbletree, bioconductor-cn.mops, bioconductor-copynumber, bioconductor-degreport, bioconductor-deseq2, bioconductor-dexseq, bioconductor-dnacopy, bioconductor-genomeinfodbdata, bioconductor-genomicranges, bioconductor-iranges, bioconductor-limma, bioconductor-purecn, bioconductor-rtracklayer, bioconductor-titancna, r-base, r-basejump=0.7.2, r-bcbiornaseq>=0.2.7, r-cghflasso, r-chbutils, r-devtools, r-dplyr, r-dt, r-ggdendro, r-ggplot2, r-ggrepel>=0.7, r-gplots, r-gsalib, r-knitr, r-pheatmap, r-plyr, r-pscbs, r-reshape, r-rmarkdown, r-rsqlite, r-sleuth, r-snow, r-stringi, r-viridis>=0.5, r-wasabi, r=3.5.1, xorg-libxt
# Installing into conda environment dv: deepvariant
Traceback (most recent call last):
  File "/hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/bin/bcbio_nextgen.py", line 221, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 90, in upgrade_bcbio
    upgrade_thirdparty_tools(args, REMOTES)
  File "/hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 495, in upgrade_thirdparty_tools
    cbl_conda.install_in(_get_conda_bin(), args.tooldir, package_yaml)
  File "/hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/tmpbcbio-install/cloudbiolinux/cloudbio/package/conda.py", line 90, in install_in
    "{py_version} {extra_pins} {pkgs_str}".format(**locals()), shell=True)
  File "/hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'export BOTO_CONFIG=/ignoreglobal && /hpf/largeprojects/ccmbio/naumenko/tools/bcbio_1.1.5/anaconda/bin/conda install -y -n dv -c conda-forge -c bioconda python=2 'openssl=1.1.1b' 'deepvariant'' returned non-zero exit status 1.

SN

roryk commented 5 years ago

Thanks, this has happened to us too, I don't understand why it is happening but Brad broke down what is going on here: https://github.com/bcbio/bcbio-nextgen/issues/2613

roryk commented 5 years ago

Closing this since it's a duplicate and reopened the #2613 issue.