planemo / conda issues - Githubissues

Background

Until september, we were able to manage functional tests and docker building using some wonderful Conda dependencies. Since, there was a huge migration to R-3.3.1 https://github.com/bioconda/bioconda-recipes/issues/2404 All our tools passed this migration. But since, there are conflicts between tools versions and channels and whatever which induce that some tools come with the R 3.2.2 and some with R 3.3.1: https://github.com/galaxyproject/planemo/issues/604 An update of Conda within Galaxy should solve this issue...

Currently, I'm testing as suggested here https://github.com/galaxyproject/tools-iuc/issues/1071 to use the last version of miniconda : miniconda3-4.2.12

miniconda installation

wget -q --recursive 'https://repo.continuum.io/miniconda/Miniconda3-4.2.12-Linux-x86_64.sh'
bash 'Miniconda3-4.2.12-Linux-x86_64.sh' -b -p /tmp/mc3-4.2.12/

deps installation

planemo conda_install --conda_prefix /tmp/mc3-4.2.12/ .

👍 Good news: all tools are installed with their 3.3.1 version

planemo test

planemo test --install_galaxy --conda_dependency_resolution --conda_prefix /tmp/mc3-4.2.12/

galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:28,567 Executing command: /tmp/mc3-4.2.12/bin/conda list --name __r-snow@0.4_1 --export > /tmp/tmpjhqeXz/tmp/jobdepsdf01NEa817a1d312c412c739d4c357c9343718c2068f458808f083afdf1f251d82564c/__r-snow@0.4_1
requests.packages.urllib3.connectionpool INFO 2016-12-21 15:53:29,172 Starting new HTTP connection (1): localhost
requests.packages.urllib3.connectionpool DEBUG 2016-12-21 15:53:29,436 "GET /api/jobs/5729865256bc2525?key=89116108df0529eaf07c60bfbc2cd985 HTTP/1.1" 200 None
galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:29,713 Executing command: /tmp/mc3-4.2.12/bin/conda create -y --unknown --offline --prefix /tmp/tmpjhqeXz/job_working_directory/000/2/conda-env --file /tmp/tmpjhqeXz/tmp/jobdepsdf01NEa817a1d312c412c739d4c357c9343718c2068f458808f083afdf1f251d82564c/__r-snow@0.4_1 > /dev/null
requests.packages.urllib3.connectionpool INFO 2016-12-21 15:53:31,044 Starting new HTTP connection (1): localhost
requests.packages.urllib3.connectionpool DEBUG 2016-12-21 15:53:31,297 "GET /api/jobs/5729865256bc2525?key=89116108df0529eaf07c60bfbc2cd985 HTTP/1.1" 200 None
galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:32,447 Executing command: /tmp/mc3-4.2.12/bin/conda clean --tarballs -y
galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:33,450 Executing command: /tmp/mc3-4.2.12/bin/conda list --name __bioconductor-xcms@1.46.0 --export > /tmp/tmpjhqeXz/tmp/jobdeps1Qjc8_89b20b2c5915d075d4a0c07dfb65cfb04a50060d7fadd95d12895e582b914f79/__bioconductor-xcms@1.46.0
requests.packages.urllib3.connectionpool INFO 2016-12-21 15:53:34,506 Starting new HTTP connection (1): localhost
galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:34,576 Executing command: /tmp/mc3-4.2.12/bin/conda install -y --unknown --offline --prefix /tmp/tmpjhqeXz/job_working_directory/000/2/conda-env --file /tmp/tmpjhqeXz/tmp/jobdeps1Qjc8_89b20b2c5915d075d4a0c07dfb65cfb04a50060d7fadd95d12895e582b914f79/__bioconductor-xcms@1.46.0 > /dev/null
requests.packages.urllib3.connectionpool DEBUG 2016-12-21 15:53:34,717 "GET /api/jobs/5729865256bc2525?key=89116108df0529eaf07c60bfbc2cd985 HTTP/1.1" 200 None
galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:36,610 Executing command: /tmp/mc3-4.2.12/bin/conda clean --tarballs -y
galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:37,592 Executing command: /tmp/mc3-4.2.12/bin/conda list --name __r-batch@1.1_4 --export > /tmp/tmpjhqeXz/tmp/jobdepsHNeXMre7325fdd48bc9d36b864284fa0ff5f09769060a0bf098ef4ab24c33996193d03/__r-batch@1.1_4
galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:38,540 Executing command: /tmp/mc3-4.2.12/bin/conda install -y --unknown --offline --prefix /tmp/tmpjhqeXz/job_working_directory/000/2/conda-env --file /tmp/tmpjhqeXz/tmp/jobdepsHNeXMre7325fdd48bc9d36b864284fa0ff5f09769060a0bf098ef4ab24c33996193d03/__r-batch@1.1_4 > /dev/null
...

UnsatisfiableError: The following specifications were found to be in conflict:
  - bzip2 1.0.6 3
Use "conda info <package>" to see the dependencies for each package.

galaxy.tools.deps.conda_util DEBUG 2016-12-21 15:53:39,769 Executing command: /tmp/mc3-4.2.12/bin/conda clean --tarballs -y
galaxy.jobs.runners ERROR 2016-12-21 15:53:40,699 (2) Failure preparing job
Traceback (most recent call last):
  File "/tmp/tmpjhqeXz/galaxy-dev/lib/galaxy/jobs/runners/__init__.py", line 170, in prepare_job
    job_wrapper.prepare()
  File "/tmp/tmpjhqeXz/galaxy-dev/lib/galaxy/jobs/__init__.py", line 913, in prepare
    self.dependency_shell_commands = self.tool.build_dependency_shell_commands(job_directory=self.working_directory)
  File "/tmp/tmpjhqeXz/galaxy-dev/lib/galaxy/tools/__init__.py", line 1331, in build_dependency_shell_commands
    tool_instance=self
  File "/tmp/tmpjhqeXz/galaxy-dev/lib/galaxy/tools/deps/__init__.py", line 104, in dependency_shell_commands
    return [dependency.shell_commands(requirement) for requirement, dependency in requirement_to_dependency.items()]
  File "/tmp/tmpjhqeXz/galaxy-dev/lib/galaxy/tools/deps/resolvers/conda.py", line 245, in shell_commands
    self.build_environment()
  File "/tmp/tmpjhqeXz/galaxy-dev/lib/galaxy/tools/deps/resolvers/conda.py", line 240, in build_environment
    raise DependencyException("Conda dependency seemingly installed but failed to build job environment.")
DependencyException: Conda dependency seemingly installed but failed to build job environment.

Investigation

/tmp/mc3-4.2.12/bin/conda list --name __r-snow@0.4_1 | grep bzip2
bzip2                     1.0.6                         3
/tmp/mc3-4.2.12/bin/conda list --name __bioconductor-xcms@1.46.0 | grep bzip2
bzip2                     1.0.6                         3
/tmp/mc3-4.2.12/bin/conda list --name __r-batch@1.1_4 | grep bzip2
bzip2                     1.0.6                         3

😭

workflow4metabolomics / tools-metabolomics

planemo / conda issues #26