GalaxyKickStart compatible with Ubuntu 16.04 (xenial)?

[remimarenco]

Hi!

Thanks for this awesome ansible playbook. It hugely streamline the installation process.

I have an issue when I want to run it on my remote machine, because of my Ubuntu 16.04 and nginx?

Here is what I run: ansible-playbook --inventory-file inventory_files/galaxy-kickstart galaxy.yml with galaxy-kickstart being configured to point to my remote server.

It is working fine until it hits this dependency error:

ASK [galaxyprojectdotorg.galaxy-os : Install required system packages] **** failed: [34.196.158.130] (item=['ant', 'cmake', 'curl', 'g++', 'gcc', 'gfortran', 'git-core', 'libffi-dev', 'liblapack-dev', 'libncurses5-dev', 'libopenblas-dev', 'libpam0g-dev', 'libpq-dev', 'libsparsehash-dev', 'make', 'mercurial', 'nginx-extras', 'openssh-server', 'patch', 'pkg-config', 'postgresql', 'postgresql', 'postgresql-client', 'python-boto', 'python-dev', 'python-prettytable', 'python-psycopg2', 'python-virtualenv', 'python-pip', 'rsync', 'samtools', 'slurm-drmaa-dev', 'supervisor', 'swig', 'sysstat', 'unzip', 'wget', 'zlib1g-dev']) => {"cache_update_time": 1494067027, "cache_updated": false, "failed": true, "item": ["ant", "cmake", "curl", "g++", "gcc", "gfortran", "git-core", "libffi-dev", "liblapack-dev", "libncurses5-dev", "libopenblas-dev", "libpam0g-dev", "libpq-dev", "libsparsehash-dev", "make", "mercurial", "nginx-extras", "openssh-server", "patch", "pkg-config", "postgresql", "postgresql", "postgresql-client", "python-boto", "python-dev", "python-prettytable", "python-psycopg2", "python-virtualenv", "python-pip", "rsync", "samtools", "slurm-drmaa-dev", "supervisor", "swig", "sysstat", "unzip", "wget", "zlib1g-dev"], "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\" install 'ant' 'cmake' 'gfortran' 'git-core' 'liblapack-dev' 'libncurses5-dev' 'libopenblas-dev' 'libpam0g-dev' 'libpq-dev' 'libsparsehash-dev' 'mercurial' 'nginx-extras' 'pkg-config' 'postgresql' 'postgresql' 'postgresql-client' 'python-boto' 'python-prettytable' 'python-psycopg2' 'samtools' 'slurm-drmaa-dev' 'supervisor' 'swig' 'sysstat' -o APT::Install-Recommends=no' failed: E: Unable to correct problems, you have held broken packages.\n", "stderr": "E: Unable to correct problems, you have held broken packages.\n", "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSome packages could not be installed. This may mean that you have\nrequested an impossible situation or if you are using the unstable\ndistribution that some required packages have not yet been created\nor been moved out of Incoming.\nThe following information may help to resolve the situation:\n\nThe following packages have unmet dependencies:\n nginx-extras : Depends: nginx-common (= 1.10.0-0ubuntu0.16.04.4ppa1) but 1.10.3-0+xenial0 is to be installed\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Some packages could not be installed. This may mean that you have", "requested an impossible situation or if you are using the unstable", "distribution that some required packages have not yet been created", "or been moved out of Incoming.", "The following information may help to resolve the situation:", "", "The following packages have unmet dependencies:", " nginx-extras : Depends: nginx-common (= 1.10.0-0ubuntu0.16.04.4ppa1) but 1.10.3-0+xenial0 is to be installed"]}

Apparently, I should modify the nginx dependencies, but not sure about the process to properly do it (or if even we can do something about it).

Thanks!

[mvdbeek]

We'll have a look at this. @drosofff I am automatically updating all ubuntu versions (14.04-16.10) with the nginx upload patch every day, so that should work and we don't need to pin nginx anymore. Do you already have a work in progress on 16.04 support?

[drosofff]

Hi @remimarenco

YES, the GKS playbook has not been extensively tested on Ubuntu 16.04. I know that there are some adjustments to be done and I am working on it. However, as you noticed it the error come from installation of the nginx patched package from @mvdbeek in the upstream galaxyprojectdotorg.galaxy-os role. I am a bit surprised because when I tested this role with 16.04 six week ago, it worked.

@mvdbeek yes I have a lazy local branch for adapting GKS to 16.04 but it's only focused on postgresql up to now. Nothing that justify to push in the repo. Feel free if you have something to push, I'll easily adapt my few changes !

@remimarenco thanks for bringing this to our attention, it will motivate us to move toward 16.04 ;-)

[drosofff]

@remimarenco Just to follow up:

I tested the GKS playbook galaxy.yml in

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:    16.04
Codename:   xenial

It's running until the galaxyprojectdotorg.galaxy-extras role which crashed at the TASK [galaxyprojectdotorg.galaxy-extras : Install SLURM system packages] task.

This is well after the step where you have an issue.

[drosofff]

@remimarenco can I close the issue ?

[colindaven]

I did have an issue here as well, but just because an apache was already running. Therefore nginx could not bind the relevant default port, and the installation failed - with the same error message about missing dependencies for nginx-extra.

This may have a bearing here, or it may make sense to check apache state is stopped/disabled in the playbook before installing nginx. Just my two cents.

[drosofff]

@colindaven OK, maybe you can raise the issue in the galaxy-extra role repo ? https://github.com/galaxyproject/ansible-galaxy-extras

[colindaven]

By the way, I apparently completed the 16.04 playbook. Not convinced Miniconda worked behind the proxy though, I will have to find it an test further. But generally, if I install on myIP, what is the URL which I can use to test the web interface ? I can't find variables so far for a non-default port.

I get an nginx access denied on http://myIP/

Thanks.

Here's my galaxy ps list

GalaxyKickStart# ps -aux | grep galaxy
root      6926  0.0  0.0  22512  3468 ?        S    10:10   0:00 /bin/bash -c /usr/bin/python /usr/sbin/configure_slurm.py && /usr/sbin/slurmctld -D -L /home/galaxy/galaxy/slurmctld.log
root      6928  0.0  0.0 133800  5548 ?        S    10:10   0:00 /usr/sbin/slurmd -D -L /home/galaxy/galaxy/slurmd.log
galaxy    6945  0.0  0.0 175020  8084 ?        S    10:10   0:00 nginx: worker process
galaxy    6958  0.0  0.0 470720  7180 ?        Sl   10:10   0:00 /usr/sbin/slurmctld -D -L /home/galaxy/galaxy/slurmctld.log
galaxy    8350  1.6  1.0 340288 87192 ?        S    10:20   0:01 /home/galaxy/galaxy/.venv/bin/python ./lib/galaxy/main.py -c /home/galaxy/galaxy/config/galaxy.ini --server-name=handler2 --log-file=/home/galaxy/galaxy/handler2.log
galaxy    8351  1.6  1.0 340320 87120 ?        S    10:20   0:01 /home/galaxy/galaxy/.venv/bin/python ./lib/galaxy/main.py -c /home/galaxy/galaxy/config/galaxy.ini --server-name=handler0 --log-file=/home/galaxy/galaxy/handler0.log
galaxy    8354  1.6  1.0 340280 87288 ?        S    10:20   0:01 /home/galaxy/galaxy/.venv/bin/python ./lib/galaxy/main.py -c /home/galaxy/galaxy/config/galaxy.ini --server-name=handler3 --log-file=/home/galaxy/galaxy/handler3.log
galaxy    8357  1.6  1.0 340280 87224 ?        S    10:20   0:01 /home/galaxy/galaxy/.venv/bin/python ./lib/galaxy/main.py -c /home/galaxy/galaxy/config/galaxy.ini --server-name=handler1 --log-file=/home/galaxy/galaxy/handler1.log
postgres  8376  0.1  0.2 298100 16376 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55500) idle
galaxy    8377  3.5  1.5 380272 127564 ?       S    10:20   0:02 /home/galaxy/galaxy/.venv/bin/python2 /home/galaxy/galaxy/.venv/bin/uwsgi --virtualenv /home/galaxy/galaxy/.venv --ini-paste /home/galaxy/galaxy/config/galaxy.ini --logdate --master --processes 1 --threads 2 --logto /home/galaxy/galaxy/uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191
postgres  8378  0.1  0.2 298096 16368 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55502) idle
postgres  8382  0.1  0.2 298096 16368 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55504) idle
postgres  8385  0.0  0.1 298108 15476 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55506) idle
postgres  8386  0.0  0.1 298108 15476 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55508) idle
postgres  8387  0.1  0.1 297968 16240 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55510) idle
postgres  8388  0.0  0.1 298108 15472 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55516) idle
postgres  8392  0.0  0.1 298108 15468 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55520) idle
postgres  8397  0.1  0.1 297968 16244 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55524) idle
postgres  8398  0.0  0.1 298108 15472 ?        Ss   10:20   0:00 postgres: galaxy galaxy 127.0.0.1(55526) idle
root      8414  0.0  0.0  15752   944 pts/2    S+   10:21   0:00 grep --color=auto galaxy

[drosofff]

@colindaven

I get an nginx access denied on http://myIP/

did you try http://localhost ?

[colindaven]

Yep, it didn't work either. Sure it's part of a bigger issue though - now struggling with the proxy again today following the latest commits and recloning GKS.

[colindaven]

I didn't solve the nginx issues on that machine. However, I retried the whole process on my (hardware identical) local machine and now have access to the web interface. So, great, it does work. Thanks for all the help guys!

[remimarenco]

Thanks for the work on this! We can close it as soon as you want

[drosofff]

Sure. Thanks for the incentive on 16.04. One had to do it !