installation error: bcbio_nextgen 1.1.4

[parlar]

Tried to install using the latest stable version of bcbio-nextgen but it exists with an error. See further down below for complete error message.

System is Centos 7.6.1810

Installation of bioconductor-genomeinfodbdata-1.2.0 exits with the following error:

ERROR conda.core.link:_execute_post_link_actions(658): An error occurred while installing package 'bioconda::bioconductor-genomeinfodbdata-1.2.0-r351_0'.
LinkError: post-link script failed for package bioconda::bioconductor-genomeinfodbdata-1.2.0-r351_0
running your command again with `-v` will provide additional information
location of failed script: /home/pela09/bcbio/anaconda/bin/.bioconductor-genomeinfodbdata-post-link.sh

Have tried to install this package from a different bioconda label but the same error persists. Have also downgraded Python to 3.6.7 but that did not solve anything.

Complete error msg:

Downloading and Extracting Packages
bioconductor-genomei | 6 KB      | ############################################################################################################################# | 100% 
r-nlme-3.1_138       | 2.3 MB    | ############################################################################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: / b'To use this package you will need to use the included training sets.\nThese can be found at /home/pela09/bcbio/anaconda/share/genesplicer-1.0-1\n\nExample for human samples:  genesplicer <fasta-file> /home/pela09/bcbio/anaconda/share/genesplicer-1.0-1/human\n\nFor more information, see https://ccb.jhu.edu/software/genesplicer/\n'
| b'\n========================= Novoalign information ===========================================\nNovoalign is installed. Documentation installed in\n/home/pela09/bcbio/anaconda/share/doc/novoalign\n\nCommercial use requires a license; contact sales@novocraft.com.\nLicense also adds multi-threading and other features.\nOnce you have a license file, run novoalign-license-register to install it.\n\nLicensing information from Novocraft website:\n--------------------------------------------------------------------------------------------\nNovoalign with some limitations is free for use by non-profit organisations. \nCommercial users should contact sales@novocraft.com to purchase a license.\nA license enables \n* Multi-threading\n* Gzipped input files\n* Amplicon mode\nand much more.\nYou must include a copy of this license if you share any Novocraft programs.\n======================== End of novoalign information ======================================\n'
| ERROR conda.core.link:_execute_post_link_actions(658): An error occurred while installing package 'bioconda::bioconductor-genomeinfodbdata-1.2.0-r351_0'.
LinkError: post-link script failed for package bioconda::bioconductor-genomeinfodbdata-1.2.0-r351_0
running your command again with `-v` will provide additional information
location of failed script: /home/pela09/bcbio/anaconda/bin/.bioconductor-genomeinfodbdata-post-link.sh
==> script messages <==
<None>

Attempting to roll back.

failed
ERROR conda.core.link:_execute(568): An error occurred while installing package 'bioconda::bioconductor-genomeinfodbdata-1.2.0-r351_0'.
LinkError: post-link script failed for package bioconda::bioconductor-genomeinfodbdata-1.2.0-r351_0
running your command again with `-v` will provide additional information
location of failed script: /home/pela09/bcbio/anaconda/bin/.bioconductor-genomeinfodbdata-post-link.sh
==> script messages <==
<None>

Attempting to roll back.

Rolling back transaction: done

LinkError: post-link script failed for package bioconda::bioconductor-genomeinfodbdata-1.2.0-r351_0
running your command again with `-v` will provide additional information
location of failed script: /home/pela09/bcbio/anaconda/bin/.bioconductor-genomeinfodbdata-post-link.sh
==> script messages <==
<None>

Traceback (most recent call last):
  File "/home/pela09/bcbio/anaconda/bin/bcbio_nextgen.py", line 221, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/home/pela09/bcbio/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 89, in upgrade_bcbio
    upgrade_thirdparty_tools(args, REMOTES)
  File "/home/pela09/bcbio/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 494, in upgrade_thirdparty_tools
    cbl_conda.install_in(_get_conda_bin(), args.tooldir, package_yaml)
  File "/home/pela09/Downloads/tmpbcbio-install/cloudbiolinux/cloudbio/package/conda.py", line 90, in install_in
    "{py_version} {extra_pins} {pkgs_str}".format(**locals()), shell=True)
  File "/home/pela09/bcbio/anaconda/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '/home/pela09/bcbio/anaconda/bin/conda install -y  -c conda-forge -c bioconda python=3 'openssl=1.1.1b' 'age-metasv' 'anaconda-client' 'awscli' 'bamtools=2.4.0' 'bamutil' 'bbmap' 'bcbio-prioritize' 'bcbio-variation' 'bcbio-variation-recall' 'bcftools' 'bedtools=2.27.1' 'bio-vcf' 'biobambam' 'bioconductor-annotate' 'bioconductor-biocgenerics' 'bioconductor-biocinstaller' 'bioconductor-biocstyle' 'bioconductor-biostrings' 'bioconductor-biovizbase' 'bioconductor-bsgenome.hsapiens.ucsc.hg19' 'bioconductor-bsgenome.hsapiens.ucsc.hg38' 'bioconductor-bubbletree' 'bioconductor-cn.mops' 'bioconductor-copynumber' 'bioconductor-degreport' 'bioconductor-deseq2' 'bioconductor-dexseq' 'bioconductor-dnacopy' 'bioconductor-genomeinfodbdata' 'bioconductor-genomicranges' 'bioconductor-iranges' 'bioconductor-limma' 'bioconductor-purecn' 'bioconductor-rtracklayer' 'bioconductor-titancna' 'bowtie' 'bowtie2' 'break-point-inspector' 'bwa' 'bwakit' 'bzip2' 'cage' 'cancerit-allelecount' 'chipseq-greylist' 'cnvkit' 'coincbc' 'cramtools' 'cufflinks' 'cyvcf2' 'deeptools' 'delly' 'duphold>=0.0.9' 'ensembl-vep=95.*' 'express' 'extract-sv-reads' 'fastp' 'fastqc>=0.11.8=1' 'fgbio' 'freebayes=1.1.0.46' 'gatk' 'gatk4' 'geneimpacts' 'genesplicer' 'gffcompare' 'goleft' 'grabix' 'gridss' 'gsort' 'gvcfgenotyper' 'h5py' 'hmftools-amber' 'hmftools-cobalt' 'hmftools-purple' 'hmmlearn' 'hts-nim-tools' 'htslib' 'impute2' 'kallisto>=0.43.1' 'kraken' 'lofreq' 'maxentscan' 'mbuffer' 'minimap2' 'mintmap' 'mirdeep2=2.0.0.7' 'mirtop' 'moreutils' 'mosdepth' 'multiqc' 'multiqc-bcbio' 'ncurses' 'ngs-disambiguate' 'nodejs' 'novoalign' 'octopus>=0.5.1b' 'oncofuse' 'p7zip' 'parallel' 'pbgzip' 'perl-app-cpanminus' 'perl-archive-extract' 'perl-archive-zip' 'perl-bio-db-sam' 'perl-cgi' 'perl-dbi' 'perl-encode-locale' 'perl-file-fetch' 'perl-file-sharedir' 'perl-file-sharedir-install' 'perl-ipc-system-simple' 'perl-lwp-protocol-https' 'perl-lwp-simple' 'perl-sanger-cgp-battenberg' 'perl-statistics-descriptive' 'perl-time-hires' 'perl-vcftools-vcf' 'picard' 'pindel' 'pizzly' 'preseq' 'pyloh' 'pysam>=0.14.0' 'pythonpy' 'qsignature' 'qualimap' 'r-base' 'r-basejump=0.7.2' 'r-bcbiornaseq>=0.2.7' 'r-cghflasso' 'r-chbutils' 'r-devtools' 'r-dplyr' 'r-dt' 'r-ggdendro' 'r-ggplot2' 'r-ggrepel>=0.7' 'r-gplots' 'r-gsalib' 'r-knitr' 'r-pheatmap' 'r-plyr' 'r-pscbs' 'r-reshape' 'r-rmarkdown' 'r-rsqlite' 'r-sleuth' 'r-snow' 'r-stringi' 'r-viridis>=0.5' 'r-wasabi' 'r=3.5.1' 'rapmap' 'razers3=3.5.0' 'readline' 'rtg-tools' 's3gof3r' 'sailfish' 'salmon' 'sambamba' 'samblaster' 'samtools' 'scalpel' 'seq2c' 'seqbuster' 'seqcluster' 'seqtk' 'sickle-trim' 'simple_sv_annotation' 'singlecell-barcodes' 'snap-aligner=1.0dev.97' 'snpeff=4.3.1t' 'solvebio' 'spades' 'star=2.6.1d' 'stringtie' 'subread' 'survivor' 'tdrmapper' 'tophat-recondition' 'ucsc-bedgraphtobigwig' 'ucsc-bedtobigbed' 'ucsc-bigbedinfo' 'ucsc-bigbedsummary' 'ucsc-bigbedtobed' 'ucsc-bigwiginfo' 'ucsc-bigwigsummary' 'ucsc-bigwigtobedgraph' 'ucsc-bigwigtowig' 'ucsc-fatotwobit' 'ucsc-gtftogenepred' 'ucsc-liftover' 'ucsc-wigtobigwig' 'vardict' 'vardict-java' 'variantbam' 'varscan' 'vcfanno' 'vcflib' 'verifybamid2' 'viennarna' 'vqsr_cnn' 'vt' 'wham' 'xorg-libxt' 'xz'' returned non-zero exit status 1.

[parlar]

seems that this is the problem.

https://github.com/bioconda/bioconda-recipes/issues/13846

[parlar]

Tried again on an Ubuntu 18.04 machine outside the corporate network, then it works. Certificate issue?

[parlar]

One more piece of information: the conda installation of bioconductor-genomeinfodbdata works if I create a separate environment but not when installing into the bcbio base environment. Wierd.

EDIT: I may have used -c biobuild instead of -c bioconda when it worked. When verifying it did not..

[chapmanb]

Pär; Thanks for the report and sorry about the issues. My suggestion for debugging is to try running the problem script outside of conda:

/home/pela09/bcbio/anaconda/bin/.bioconductor-genomeinfodbdata-post-link.sh

This is a shell script that actually does the install download. Since the data files for R in this case are large, it isn't directly in the bioconda package but rather part of this downstream step. It sounds like there is something in your network environment blocking this but it's hard to know what from just the error message. Hopefully running it separately will reveal some clues to help debug and avoid. Thanks much for the report and help debugging.

[parlar]

I should give an update on this:

The error is due to that the ca-certificate path is not correctly set. Even if I set ca-certificates explicitly in conda it fails. I have entered an issue here: https://github.com/conda/conda/issues/8653 .

Now the bioconductor-genomeinfodbdata package has been modified to use curl (instead of wget) but the problem remains since it's due to conda not seeing certificates. But by setting export SSL_CERT_FILE=/path/to/ca.pem, however, the package can be installed and then everything works.

It it has nothing todo with bioconductor or bcbio, but I thought I should make an entry here if anyone else encounters this issue.

[parlar]

Another update:

mingwandroid wrote (https://github.com/conda/conda/issues/8653) "Conda's ssl settings pertain only to conda itself and not to and other packages you can install via conda."

This means that the installation will always fail if you're inside a corporate network that depends on a correctly set ca-cert. I have proposed here https://github.com/bioconda/bioconda-recipes/issues/13846 that the conda package installation is modified to set ca-cert for curl if it's set in conda. Don't know if this is the right way of fixing it?

[chapmanb]

Pär; Thanks for investigating this. It sounds like the export SSL_CERT approach to point at the valid certification for your environment is a work around, is that right? If so, it seems like the best approach to avoid these and we could add a note to the documentation so folks know to look at that as an option if they're in a restricted environment.

[parlar]

If the SSL_CERT_FILE env variable is still visible from within the conda env I guess it must work. I can double check on Monday.

[roryk]

Thanks, let us know if that ends up working so we can add in the documentation. Nice one finding a possible workaround.