search

BackofenLab / GraphClust-2

A pipeline for structural clustering of RNA secondary structures
GNU General Public License v3.0
14 stars 4 forks source link

conda issue for dependencies #53

Closed mmiladi closed 6 years ago

mmiladi commented 6 years ago

Recent docker builds are missing binary files for at least two dependencies (r-scape and graphicsmagic) in the conda environment.

root@d78eb3aeff9e:/export/tool_deps/_conda/envs/mulled-v1-7cc55d4faf2226e75728e1595b08d0f7cdf9df9e42ca7b72736337c5adc8f30b/bin# ls gm
ls: cannot access gm: No such file or directory

The tools are reported to be installed in the admin section: https://files.gitter.im/mmiladi/naMU/Screenshot-from-2017-11-08-19-47-54.png

The docker built reports some warnings that might be related, but unfort I couldn't trace it https://hub.docker.com/r/backofenlab/docker-galaxy-graphclust/builds/bgm632if4ep6ooflhhnjxa5/

From the docker log:

..
Removing intermediate container 78e8d6d4eb41

Step 7/12 : ADD ./tours/* $GALAXY_ROOT/config/plugins/tours/

 ---> cb571ff63108

Removing intermediate container 1c7e2be5f860

Step 8/12 : ADD library_data.yaml $GALAXY_ROOT/library_data.yaml

 ---> fefc990320b4

Removing intermediate container 37ad2222c8dc

Step 9/12 : ADD ./workflows/* $GALAXY_ROOT/workflows/

 ---> 00edc96b285e

Removing intermediate container 82208dbb89af

Step 10/12 : RUN startup_lite &&     sleep 30 &&     . $GALAXY_VIRTUAL_ENV/bin/activate &&     workflow-install --workflow_path $GALAXY_ROOT/workflows/ -g http://localhost:8080 -u $GALAXY_DEFAULT_ADMIN_USER -p $GALAXY_DEFAULT_ADMIN_PASSWORD &&     setup-data-libraries -i $GALAXY_ROOT/library_data.yaml -g http://localhost:8080 -u $GALAXY_DEFAULT_ADMIN_USER -p $GALAXY_DEFAULT_ADMIN_PASSWORD

 ---> Running in abe7b492197d

 * Starting PostgreSQL 9.3 database server

   ...done.

Activating virtualenv at /galaxy_venv

The directory '/home/galaxy/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

The directory '/home/galaxy/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Requirement already satisfied: pip>=8.1 in /galaxy_venv/lib/python2.7/site-packages

/galaxy_venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/galaxy_venv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning

Activating virtualenv at /galaxy_venv

galaxy.tools.deps DEBUG 2017-11-09 10:23:40,830 Unable to find config file './dependency_resolvers_conf.xml'

 ---> bf2b502747a0

Removing intermediate container abe7b492197d

Step 11/12 : ADD workflow_early.png $GALAXY_CONFIG_DIR/web/welcome_image.png

..
mmiladi commented 6 years ago

It would be probably better to directly jump to 17,09 by fixing issue https://github.com/BackofenLab/docker-galaxy-graphclust/issues/54 and comeback to this issue only if 17.09 faced same error.

mmiladi commented 6 years ago 
=> /home/galaxy/logs/handler1.log <==
galaxy.jobs.handler DEBUG 2017-11-10 15:29:31,803 (33) Dispatching to slurm runner
galaxy.jobs DEBUG 2017-11-10 15:29:31,838 (33) Persisting job destination (destination id: slurm_cluster)
galaxy.jobs.runners DEBUG 2017-11-10 15:29:31,849 Job [33] queued (46.285 ms)
galaxy.jobs.handler INFO 2017-11-10 15:29:31,865 (33) Job dispatched
galaxy.tools.deps DEBUG 2017-11-10 15:29:32,025 Using dependency infernal version None of type conda
galaxy.tools.deps DEBUG 2017-11-10 15:29:32,025 Using dependency infernal version 1.1 of type conda
galaxy.tools.deps.conda_util DEBUG 2017-11-10 15:29:32,026 Executing command: /export/tool_deps/_conda/bin/conda list --name __infernal@_uv_ --export > /export/galaxy-central/database/files/jobdeps2Dz5jw30a4162512b802df921aa4f183ce64d36eef064a290e5c00aad513d951be3964/__infernal@_uv_
galaxy.tools.deps.conda_util DEBUG 2017-11-10 15:29:32,856 Executing command: /export/tool_deps/_conda/bin/conda create -y --override-channels --channel iuc --channel bioconda --channel conda-forge --channel defaults --channel r --unknown --offline --prefix /export/galaxy-central/database/job_working_directory/000/33/conda-env --file /export/galaxy-central/database/files/jobdeps2Dz5jw30a4162512b802df921aa4f183ce64d36eef064a290e5c00aad513d951be3964/__infernal@_uv_ > /dev/null

==> /home/galaxy/logs/uwsgi.log <==
132.230.153.74 - - [10/Nov/2017:15:29:35 +0000] "GET /api/histories/f597429621d6eb2b/contents?details=5a9381f02a7b89af&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-11-10T15%3A29%3A31.000Z&qv=False&qv=False HTTP/1.1" 200 - "http://192.52.2.195:8888/?job_id=d413a19dec13d11e&__identifer=k2xhezh3vx8" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
[pid: 266|app: 0|req: 78/192] 132.230.153.74 () {44 vars in 1172 bytes} [Fri Nov 10 15:29:35 2017] GET /api/histories/f597429621d6eb2b/contents?details=5a9381f02a7b89af&order=hid&v=dev&q=update_time-ge&q=deleted&q=purged&qv=2017-11-10T15%3A29%3A31.000Z&qv=False&qv=False => generated 2 bytes in 49 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 0)

==> /home/galaxy/logs/handler1.log <==
galaxy.tools.deps.conda_util DEBUG 2017-11-10 15:29:37,103 Executing command: /export/tool_deps/_conda/bin/conda clean --tarballs -y
galaxy.jobs.runners ERROR 2017-11-10 15:29:37,878 (33) Failure preparing job
Traceback (most recent call last):
  File "/galaxy-central/lib/galaxy/jobs/runners/__init__.py", line 170, in prepare_job
    job_wrapper.prepare()
  File "/galaxy-central/lib/galaxy/jobs/__init__.py", line 886, in prepare
    self.dependency_shell_commands = self.tool.build_dependency_shell_commands(job_directory=self.working_directory)
  File "/galaxy-central/lib/galaxy/tools/__init__.py", line 1450, in build_dependency_shell_commands
    tool_instance=self
  File "/galaxy-central/lib/galaxy/tools/deps/__init__.py", line 113, in dependency_shell_commands
    return [dependency.shell_commands(requirement) for requirement, dependency in requirement_to_dependency.items()]
  File "/galaxy-central/lib/galaxy/tools/deps/resolvers/conda.py", line 470, in shell_commands
    self.build_environment()
  File "/galaxy-central/lib/galaxy/tools/deps/resolvers/conda.py", line 465, in build_environment
    raise DependencyException("Conda dependency seemingly installed but failed to build job environment.")
DependencyException: Conda dependency seemingly installed but failed to build job environment.
galaxy.tools.error_reports DEBUG 2017-11-10 15:29:38,025 Bug report plugin <galaxy.tools.error_reports.plugins.sentry.SentryPlugin object at 0x7ff8a5c92e90> generated response None
galaxy.model.metadata DEBUG 2017-11-10 15:29:38,027 Cleaning up external metadata files
mmiladi commented 6 years ago 
root@ce87f77ff92f:/galaxy-central# /export/tool_deps/_conda/bin/conda create -y --override-channels --channel iuc --channel bioconda --channel conda-forge --channel defaults --channel r --unknown --offline --prefix /export/galaxy-central/database/job_working_directory/000/33/conda-env --file /export/galaxy-central/database/files/jobdeps2Dz5jw30a4162512b802df921aa4f183ce64d36eef064a290e5c00aad513d951be3964/__infernal@_uv_

CondaFileIOError: '/export/galaxy-central/database/files/jobdeps2Dz5jw30a4162512b802df921aa4f183ce64d36eef064a290e5c00aad513d951be3964/__infernal@_uv_'. [Errno 2] No such file or directory: '/export/galaxy-central/database/files/jobdeps2Dz5jw30a4162512b802df921aa4f183ce64d36eef064a290e5c00aad513d951be3964/__infernal@_uv_'
mmiladi commented 6 years ago

@bgruening Is this packge renamed/removed? gnu_coreutils

https://github.com/bgruening/galaxytools/blob/5974fd51d00414bc1d2da7fde532820e86730c72/tools/rna_tools/infernal/macros.xml#L6

mmiladi commented 6 years ago

This issue is resolved with graphclust-wrappers version 0.5.1 and galaxy upgarde to 17.09