search

chapmanb / cloudbiolinux

CloudBioLinux: configure virtual (or real) machines with tools for biological analyses
http://cloudbiolinux.org
MIT License
257 stars 160 forks source link

Installation problem conda package libtiff #386

Closed rchekaluk closed 2 years ago

rchekaluk commented 2 years ago

Installing via bcbio results in error "package libtiff-4.3.0-hf544144_1 requires libdeflate >=1.7,<1.8.0a0, but none of the providers can be installed". Presumably it is related to this recent commit?

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"

Here is a summary of the libtiff aspects of the installation:


$ python3 bcbio_nextgen_install.py /usr/local/share/bcbio --tooldir=/usr/local --aligners bwa --aligners bowtie2
...

--2021-10-22 11:08:10--  https://raw.githubusercontent.com/bcbio/bcbio-nextgen/master/requirements-conda.txt
HTTP request sent, awaiting response... 200 OK
2021-10-22 11:08:10 (1.06 MB/s) - ‘requirements-conda.txt’ saved [20/20]
...

## Package Plan ##
 environment location: /usr/local/share/bcbio/anaconda
The following packages will be downloaded:
    package                    |            build
    ---------------------------|-----------------
    libtiff-4.2.0              |       h85742a9_0         502 KB
...

The following NEW packages will be INSTALLED:
  libtiff            pkgs/main/linux-64::libtiff-4.2.0-h85742a9_0
...

## Package Plan ##
  environment location: /usr/local/share/bcbio/anaconda
The following packages will be downloaded:
    package                    |            build
    ---------------------------|-----------------
    libtiff-4.3.0              |       hf544144_1         668 KB  conda-forge
...

The following packages will be UPDATED:
  libtiff               pkgs/main::libtiff-4.2.0-h85742a9_0 --> conda-forge::libtiff-4.3.0-hf544144_1
...

Encountered problems while solving:
  - package libtiff-4.3.0-hf544144_1 requires libdeflate >=1.7,<1.8.0a0, but none of the providers can be installed
...

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... Killed
Upgrading bcbio
Upgrading third party tools to latest versions
Reading packages from /home/ubuntu/tmpbcbio-install/cloudbiolinux/contrib/flavor/ngs_pipeline_minimal/packages-conda.yaml
...

Traceback (most recent call last):
  File "/home/ubuntu/tmpbcbio-install/cloudbiolinux/cloudbio/package/conda.py", line 68, in _install_env_pkgs
    "{py_version} {pkgs_str}".format(**locals()), shell=True)
  File "/usr/local/share/bcbio/anaconda/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
...

subprocess.CalledProcessError: Command '/usr/local/share/bcbio/anaconda/bin/mamba install -q -y  -c conda-forge -c bioconda python=3 'age-metasv' 'anaconda-client' 'arriba=1.2.0' 'awscli' 'bamtools=2.4.0' 'bamutil' 'bbmap' 'bcbio-prioritize' 'bcbio-variation' 'bcbio-variation-recall' 'bcftools' 'bedops' 'bedtools' 'bio-vcf' 'biobambam=2.0.87' 'bioconductor-annotate' 'bioconductor-apeglm' 'bioconductor-biocgenerics' 'bioconductor-biocinstaller' 'bioconductor-biocstyle' 'bioconductor-biostrings' 'bioconductor-biovizbase' 'bioconductor-bsgenome.hsapiens.ucsc.hg19' 'bioconductor-bsgenome.hsapiens.ucsc.hg38' 'bioconductor-bubbletree' 'bioconductor-cn.mops' 'bioconductor-copynumber' 'bioconductor-degreport' 'bioconductor-deseq2' 'bioconductor-dnacopy' 'bioconductor-genomeinfodbdata' 'bioconductor-genomicranges' 'bioconductor-iranges' 'bioconductor-limma' 'bioconductor-rtracklayer' 'bioconductor-snpchip' 'bioconductor-titancna' 'bioconductor-vsn>=3.50.0' 'bowtie' 'bowtie2' 'break-point-inspector' 'bwa' 'bzip2' 'cage' 'cancerit-allelecount' 'chipseq-greylist' 'cnvkit' 'coincbc' 'cramtools' 'cufflinks' 'cyvcf2' 'deeptools' 'delly' 'duphold' 'ensembl-vep=100.*' 'express' 'extract-sv-reads' 'fastp' 'fastqc>=0.11.8=1' 'fgbio' 'freebayes' 'gatk' 'gatk4' 'geneimpacts' 'genesplicer' 'gffcompare' 'goleft' 'grabix' 'gridss' 'gsort' 'gvcfgenotyper' 'h5py' 'hisat2' 'hmftools-amber' 'hmftools-cobalt' 'hmftools-purple' 'hmmlearn' 'hts-nim-tools' 'htslib' 'impute2' 'kallisto>=0.43.1' 'kraken' 'ldc>=1.13.0' 'lofreq' 'macs2' 'maxentscan' 'mbuffer' 'minimap2' 'mintmap' 'mirdeep2=2.0.0.7' 'mirtop' 'moreutils' 'multiqc' 'multiqc-bcbio' 'ncurses' 'ngs-disambiguate' 'nodejs' 'novoalign' 'octopus>=0.5.1b' 'oncofuse' 'optitype>=1.3.4' 'p7zip' 'pandoc=2.9.2' 'parallel' 'pbgzip' 'peddy' 'perl-app-cpanminus' 'perl-archive-extract' 'perl-archive-zip' 'perl-bio-db-sam' 'perl-cgi' 'perl-dbi' 'perl-encode-locale' 'perl-file-fetch' 'perl-file-sharedir' 'perl-file-sharedir-install' 'perl-ipc-system-simple' 'perl-lwp-protocol-https' 'perl-lwp-simple' 'perl-sanger-cgp-battenberg' 'perl-statistics-descriptive' 'perl-time-hires' 'perl-vcftools-vcf' 'picard' 'pindel' 'pizzly' 'pyloh' 'pysam>=0.14.0' 'pythonpy' 'qsignature' 'qualimap' 'r-base' 'r-basejump=0.7.2' 'r-bcbiornaseq>=0.2.7' 'r-cghflasso' 'r-chbutils' 'r-devtools' 'r-dplyr' 'r-dt' 'r-ggdendro' 'r-ggplot2' 'r-ggrepel>=0.7' 'r-gplots' 'r-gsalib' 'r-janitor' 'r-knitr' 'r-optparse' 'r-pheatmap' 'r-plyr' 'r-pscbs' 'r-reshape' 'r-rmarkdown' 'r-rsqlite' 'r-sleuth' 'r-snow' 'r-stringi' 'r-viridis>=0.5' 'r-wasabi' 'r=3.5.1' 'rapmap' 'razers3=3.5.0' 'readline' 'rtg-tools' 's3gof3r' 'sailfish' 'salmon' 'sambamba' 'samblaster' 'samtools=1.9' 'scalpel' 'seq2c<2016' 'seqbuster' 'seqcluster' 'seqtk' 'sickle-trim' 'simple_sv_annotation' 'singlecell-barcodes' 'snap-aligner=1.0dev.97' 'snpeff=4.3.1t' 'solvebio' 'spades' 'staden_io_lib' 'star=2.6.1d' 'stringtie' 'subread' 'survivor' 'tdrmapper' 'tophat-recondition' 'trim-galore' 'ucsc-bedgraphtobigwig' 'ucsc-bedtobigbed' 'ucsc-bigbedinfo' 'ucsc-bigbedsummary' 'ucsc-bigbedtobed' 'ucsc-bigwiginfo' 'ucsc-bigwigsummary' 'ucsc-bigwigtobedgraph' 'ucsc-bigwigtowig' 'ucsc-fatotwobit' 'ucsc-gtftogenepred' 'ucsc-liftover' 'ucsc-wigtobigwig' 'umis' 'vardict-java' 'vardict<=2015' 'variantbam' 'varscan' 'vcfanno' 'vcflib' 'verifybamid2' 'viennarna' 'vqsr_cnn' 'vt' 'wham' 'xorg-libxt' 'xz'' returned non-zero exit status 1.
...

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/share/bcbio/anaconda/bin/bcbio_nextgen.py", line 228, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/usr/local/share/bcbio/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 91, in upgrade_bcbio
    upgrade_thirdparty_tools(args, REMOTES)
  File "/usr/local/share/bcbio/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 509, in upgrade_thirdparty_tools
    cbl_conda.install_in(conda_bin, args.tooldir, package_yaml)
  File "/home/ubuntu/tmpbcbio-install/cloudbiolinux/cloudbio/package/conda.py", line 120, in install_in
    conda_pkg_list = _install_env_pkgs(env_name, env_packages, conda_bin, conda_envs, channels)
  File "/home/ubuntu/tmpbcbio-install/cloudbiolinux/cloudbio/package/conda.py", line 73, in _install_env_pkgs
    "{py_version} {pkgs_str}".format(**locals()), shell=True)
  File "/usr/local/share/bcbio/anaconda/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '/usr/local/share/bcbio/anaconda/bin/conda install -q -y  -c conda-forge -c bioconda python=3 'age-metasv' 'anaconda-client' 'arriba=1.2.0' 'awscli' 'bamtools=2.4.0' 'bamutil' 'bbmap' 'bcbio-prioritize' 'bcbio-variation' 'bcbio-variation-recall' 'bcftools' 'bedops' 'bedtools' 'bio-vcf' 'biobambam=2.0.87' 'bioconductor-annotate' 'bioconductor-apeglm' 'bioconductor-biocgenerics' 'bioconductor-biocinstaller' 'bioconductor-biocstyle' 'bioconductor-biostrings' 'bioconductor-biovizbase' 'bioconductor-bsgenome.hsapiens.ucsc.hg19' 'bioconductor-bsgenome.hsapiens.ucsc.hg38' 'bioconductor-bubbletree' 'bioconductor-cn.mops' 'bioconductor-copynumber' 'bioconductor-degreport' 'bioconductor-deseq2' 'bioconductor-dnacopy' 'bioconductor-genomeinfodbdata' 'bioconductor-genomicranges' 'bioconductor-iranges' 'bioconductor-limma' 'bioconductor-rtracklayer' 'bioconductor-snpchip' 'bioconductor-titancna' 'bioconductor-vsn>=3.50.0' 'bowtie' 'bowtie2' 'break-point-inspector' 'bwa' 'bzip2' 'cage' 'cancerit-allelecount' 'chipseq-greylist' 'cnvkit' 'coincbc' 'cramtools' 'cufflinks' 'cyvcf2' 'deeptools' 'delly' 'duphold' 'ensembl-vep=100.*' 'express' 'extract-sv-reads' 'fastp' 'fastqc>=0.11.8=1' 'fgbio' 'freebayes' 'gatk' 'gatk4' 'geneimpacts' 'genesplicer' 'gffcompare' 'goleft' 'grabix' 'gridss' 'gsort' 'gvcfgenotyper' 'h5py' 'hisat2' 'hmftools-amber' 'hmftools-cobalt' 'hmftools-purple' 'hmmlearn' 'hts-nim-tools' 'htslib' 'impute2' 'kallisto>=0.43.1' 'kraken' 'ldc>=1.13.0' 'lofreq' 'macs2' 'maxentscan' 'mbuffer' 'minimap2' 'mintmap' 'mirdeep2=2.0.0.7' 'mirtop' 'moreutils' 'multiqc' 'multiqc-bcbio' 'ncurses' 'ngs-disambiguate' 'nodejs' 'novoalign' 'octopus>=0.5.1b' 'oncofuse' 'optitype>=1.3.4' 'p7zip' 'pandoc=2.9.2' 'parallel' 'pbgzip' 'peddy' 'perl-app-cpanminus' 'perl-archive-extract' 'perl-archive-zip' 'perl-bio-db-sam' 'perl-cgi' 'perl-dbi' 'perl-encode-locale' 'perl-file-fetch' 'perl-file-sharedir' 'perl-file-sharedir-install' 'perl-ipc-system-simple' 'perl-lwp-protocol-https' 'perl-lwp-simple' 'perl-sanger-cgp-battenberg' 'perl-statistics-descriptive' 'perl-time-hires' 'perl-vcftools-vcf' 'picard' 'pindel' 'pizzly' 'pyloh' 'pysam>=0.14.0' 'pythonpy' 'qsignature' 'qualimap' 'r-base' 'r-basejump=0.7.2' 'r-bcbiornaseq>=0.2.7' 'r-cghflasso' 'r-chbutils' 'r-devtools' 'r-dplyr' 'r-dt' 'r-ggdendro' 'r-ggplot2' 'r-ggrepel>=0.7' 'r-gplots' 'r-gsalib' 'r-janitor' 'r-knitr' 'r-optparse' 'r-pheatmap' 'r-plyr' 'r-pscbs' 'r-reshape' 'r-rmarkdown' 'r-rsqlite' 'r-sleuth' 'r-snow' 'r-stringi' 'r-viridis>=0.5' 'r-wasabi' 'r=3.5.1' 'rapmap' 'razers3=3.5.0' 'readline' 'rtg-tools' 's3gof3r' 'sailfish' 'salmon' 'sambamba' 'samblaster' 'samtools=1.9' 'scalpel' 'seq2c<2016' 'seqbuster' 'seqcluster' 'seqtk' 'sickle-trim' 'simple_sv_annotation' 'singlecell-barcodes' 'snap-aligner=1.0dev.97' 'snpeff=4.3.1t' 'solvebio' 'spades' 'staden_io_lib' 'star=2.6.1d' 'stringtie' 'subread' 'survivor' 'tdrmapper' 'tophat-recondition' 'trim-galore' 'ucsc-bedgraphtobigwig' 'ucsc-bedtobigbed' 'ucsc-bigbedinfo' 'ucsc-bigbedsummary' 'ucsc-bigbedtobed' 'ucsc-bigwiginfo' 'ucsc-bigwigsummary' 'ucsc-bigwigtobedgraph' 'ucsc-bigwigtowig' 'ucsc-fatotwobit' 'ucsc-gtftogenepred' 'ucsc-liftover' 'ucsc-wigtobigwig' 'umis' 'vardict-java' 'vardict<=2015' 'variantbam' 'varscan' 'vcfanno' 'vcflib' 'verifybamid2' 'viennarna' 'vqsr_cnn' 'vt' 'wham' 'xorg-libxt' 'xz'' returned non-zero exit status 137.
Checking required dependencies
Installing isolated base python installation
Installing conda
Installing conda-build
Installing bcbio-nextgen
Installing data and third party dependencies
Traceback (most recent call last):
  File "bcbio_nextgen_install.py", line 302, in <module>
    main(parser.parse_args(), sys.argv[1:])
  File "bcbio_nextgen_install.py", line 55, in main
    subprocess.check_call([bcbio, "upgrade"] + _clean_args(sys_argv, args))
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/local/share/bcbio/anaconda/bin/bcbio_nextgen.py', 'upgrade', '--tooldir=/usr/local', '--aligners', 'bwa', '--aligners', 'bowtie2', '--data']' returned non-zero exit status 1.
naumenko-sa commented 2 years ago

Thanks for this commit, it will be useful to those who want to pull the particular cloudbiolinux version!

However, it won't solve this current issue. The solution here is likely to comb the packages and get rid of old r3.5.1 dependencies, which started giving unsolvable environments. I am testing this right away.

Also, moving data recipes, conda package specification, and bits of code from cloudbiolinux to bcbio also seems logical in the future - synchronized versions etc.