search

bgruening / docker-galaxy

:whale::bar_chart::books: Docker Images tracking the stable Galaxy releases.
http://bgruening.github.io/docker-galaxy
MIT License
226 stars 134 forks source link

Can't update postgre database #466

Closed zslastman closed 5 years ago

zslastman commented 5 years ago

I'm on a Ubuntu 16.04 server, (trying) to run your galaxy instance.

Previously, I seem to have inadvertantly changed the version of the instance I was running, and am now getting complaints that the database needs to be updated.

I'm therefore trying to migrate to a newer version of the container, as per the instructions in 'upgrading images' in the README. However when I try to create the auxilliary docker and upgrade the database:

galaxy@GalaxyHU-server:~$ sudo docker run -it --rm -v /data/galaxy-data:/export bgruening/galaxy-stable /bin/bash
root@46942632c436:/galaxy-central# startup && sh manage_db.sh upgrade
Enable Galaxy reports authentification 
umount: /var/lib/docker: must be superuser to umount
tmpfs on /proc/kcore type tmpfs (rw,nosuid,size=65536k,mode=755)
Disable Galaxy Interactive Environments. Start with --privileged to enable IE's.
Starting postgres
postgresql: started
Checking if database is up and running
Database connected
Starting cron
cron: started
Starting ProFTP
proftpd: started
Starting Galaxy reports webapp
reports: started
Starting nodejs
galaxy:galaxy_nodejs_proxy: started
Starting condor
condor: started
Starting slurmctld
Starting slurmd
Creating admin user admin@galaxy.org with key admin and password admin if not existing
Traceback (most recent call last):
  File "/usr/local/bin/create_galaxy_user.py", line 60, in <module>
    add_user(sa_session, security_agent, options.user, options.password, key=options.key, username=options.username)
  File "/usr/local/bin/create_galaxy_user.py", line 36, in add_user
    sa_session.flush()
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/scoping.py", line 153, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2254, in flush
    self._flush(objects)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2380, in _flush
    transaction.rollback(_capture_exception=True)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2344, in _flush
    flush_context.execute()
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute
    rec.execute(self)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute
    uow
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 866, in _emit_insert_statements
    execute(statement, params)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/galaxy_venv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "ix_api_keys_key"
DETAIL:  Key (key)=(admin) already exists.
 [SQL: 'INSERT INTO api_keys (create_time, user_id, key) VALUES (%(create_time)s, %(user_id)s, %(key)s) RETURNING api_keys.id'] [parameters: {'create_time': datetime.datetime(2018, 12, 18, 11, 19, 31, 256389), 'user_id': 2, 'key': 'admin'}] (Background on this error at: http://sqlalche.me/e/gkpj)
==> /home/galaxy/logs/handler0.log <==
galaxy.web.stack DEBUG 2018-12-18 11:18:54,646 server_name set to: handler0
galaxy.queue_worker INFO 2018-12-18 11:18:54,646 Initializing handler0 Galaxy Queue Worker on sqlalchemy+postgresql://galaxy:********@localhost:5432/galaxy?client_encoding=utf8
galaxy.web.stack DEBUG 2018-12-18 11:18:57,666 server_name set to: handler0
galaxy.queue_worker INFO 2018-12-18 11:18:57,666 Initializing handler0 Galaxy Queue Worker on sqlalchemy+postgresql://galaxy:********@localhost:5432/galaxy?client_encoding=utf8
galaxy.web.stack DEBUG 2018-12-18 11:19:00,889 server_name set to: handler0
galaxy.queue_worker INFO 2018-12-18 11:19:00,889 Initializing handler0 Galaxy Queue Worker on sqlalchemy+postgresql://galaxy:********@localhost:5432/galaxy?client_encoding=utf8
galaxy.model.migrate.check INFO 2018-12-18 11:19:01,273 At database version 141
tail: unrecognized file system type 0x794c7630 for ‘/home/galaxy/logs/handler0.log’. please report this to bug-coreutils@gnu.org. reverting to polling

==> /home/galaxy/logs/handler1.log <==
galaxy.web.stack DEBUG 2018-12-18 11:18:54,646 server_name set to: handler1
galaxy.queue_worker INFO 2018-12-18 11:18:54,646 Initializing handler1 Galaxy Queue Worker on sqlalchemy+postgresql://galaxy:********@localhost:5432/galaxy?client_encoding=utf8
galaxy.web.stack DEBUG 2018-12-18 11:18:57,722 server_name set to: handler1
galaxy.queue_worker INFO 2018-12-18 11:18:57,723 Initializing handler1 Galaxy Queue Worker on sqlalchemy+postgresql://galaxy:********@localhost:5432/galaxy?client_encoding=utf8
galaxy.web.stack DEBUG 2018-12-18 11:19:00,946 server_name set to: handler1
galaxy.queue_worker INFO 2018-12-18 11:19:00,946 Initializing handler1 Galaxy Queue Worker on sqlalchemy+postgresql://galaxy:********@localhost:5432/galaxy?client_encoding=utf8
galaxy.model.migrate.check INFO 2018-12-18 11:19:01,359 At database version 141
tail: unrecognized file system type 0x794c7630 for ‘/home/galaxy/logs/handler1.log’. please report this to bug-coreutils@gnu.org. reverting to polling

==> /home/galaxy/logs/reports.log <==
galaxy.web.framework.base DEBUG 2018-12-18 11:19:05,302 Enabling 'system' controller, class: System
galaxy.web.framework.base DEBUG 2018-12-18 11:19:05,303 Enabling 'users' controller, class: Users
galaxy.web.framework.base DEBUG 2018-12-18 11:19:05,304 Enabling 'home' controller, class: HomePage
galaxy.webapps.util DEBUG 2018-12-18 11:19:05,305 Enabling 'paste.httpexceptions' middleware
galaxy.webapps.util DEBUG 2018-12-18 11:19:05,305 Enabling 'RecursiveMiddleware' middleware
galaxy.webapps.util DEBUG 2018-12-18 11:19:05,309 Enabling 'ErrorMiddleware' middleware
galaxy.webapps.util DEBUG 2018-12-18 11:19:05,310 Enabling 'TransLogger' middleware
galaxy.webapps.util DEBUG 2018-12-18 11:19:05,310 Enabling 'XForwardedHostMiddleware' middleware
Starting server in PID 378.
serving on http://127.0.0.1:9001
tail: unrecognized file system type 0x794c7630 for ‘/home/galaxy/logs/reports.log’. please report this to bug-coreutils@gnu.org. reverting to polling

==> /home/galaxy/logs/slurmctld.log <==
[2018-12-18T11:19:29.953] No trigger state file (/tmp/slurm/trigger_state.old) to recover
[2018-12-18T11:19:29.953] error: Incomplete trigger data checkpoint file
[2018-12-18T11:19:29.953] read_slurm_conf: backup_controller not specified.
[2018-12-18T11:19:29.953] Reinitializing job accounting state
[2018-12-18T11:19:29.953] cons_res: select_p_reconfigure
[2018-12-18T11:19:29.953] cons_res: select_p_node_init
[2018-12-18T11:19:29.953] cons_res: preparing for 1 partitions
[2018-12-18T11:19:29.953] Running as primary controller
[2018-12-18T11:19:29.957] error: slurm_receive_msg: Zero Bytes were transmitted or received
[2018-12-18T11:19:29.967] error: slurm_receive_msg: Zero Bytes were transmitted or received
tail: unrecognized file system type 0x794c7630 for ‘/home/galaxy/logs/slurmctld.log’. please report this to bug-coreutils@gnu.org. reverting to polling

==> /home/galaxy/logs/slurmd.log <==
[2018-12-18T11:19:29.953] Node configuration differs from hardware: CPUs=20:20(hw) Boards=1:1(hw) SocketsPerBoard=20:1(hw) CoresPerSocket=1:10(hw) ThreadsPerCore=1:2(hw)
[2018-12-18T11:19:29.954] Gathering cpu frequency information for 20 cpus
[2018-12-18T11:19:29.955] slurmd version 2.6.5 started
[2018-12-18T11:19:29.956] slurmd started on Tue, 18 Dec 2018 11:19:29 +0000
[2018-12-18T11:19:29.956] CPUs=20 Boards=1 Sockets=20 Cores=1 Threads=1 Memory=64351 TmpDisk=334463 Uptime=54357858
[2018-12-18T11:19:29.957] error: Munge encode failed: Failed to access "/var/run/munge/munge.socket.2": No such file or directory (retrying ...)
[2018-12-18T11:19:29.957] error: Munge encode failed: Failed to access "/var/run/munge/munge.socket.2": No such file or directory (retrying ...)
[2018-12-18T11:19:29.957] error: Munge encode failed: Failed to access "/var/run/munge/munge.socket.2": No such file or directory
[2018-12-18T11:19:29.957] error: authentication: Socket communication error
[2018-12-18T11:19:29.957] error: Unable to register: Protocol authentication error
tail: unrecognized file system type 0x794c7630 for ‘/home/galaxy/logs/slurmd.log’. please report this to bug-coreutils@gnu.org. reverting to polling

==> /home/galaxy/logs/uwsgi.log <==
Tue Dec 18 11:18:56 2018 - your server socket listen backlog is limited to 100 connections
Tue Dec 18 11:18:56 2018 - your mercy for graceful operations on workers is 60 seconds
Tue Dec 18 11:18:56 2018 - mapped 460128 bytes (449 KB) for 8 cores
Tue Dec 18 11:18:56 2018 - *** Operational MODE: preforking+threaded ***
Tue Dec 18 11:18:56 2018 - added lib/ to pythonpath.
Tue Dec 18 11:18:56 2018 - added lib/ to pythonpath.
DEBUG:galaxy.app:python path is: lib/, lib/, ., , /galaxy_venv/bin, /galaxy_venv/lib/python2.7, /galaxy_venv/lib/python2.7/plat-x86_64-linux-gnu, /galaxy_venv/lib/python2.7/lib-tk, /galaxy_venv/lib/python2.7/lib-old, /galaxy_venv/lib/python2.7/lib-dynload, /usr/lib/python2.7, /usr/lib/python2.7/plat-x86_64-linux-gnu, /usr/lib/python2.7/lib-tk, /galaxy_venv/local/lib/python2.7/site-packages
DEBUG:galaxy.containers:config file './config/containers_conf.yml' does not exist, running with default config
galaxy.queue_worker INFO 2018-12-18 11:18:58,491 [p:206,w:0,m:0] [MainThread] Initializing main Galaxy Queue Worker on sqlalchemy+postgresql://galaxy:********@localhost:5432/galaxy?client_encoding=utf8
galaxy.model.migrate.check INFO 2018-12-18 11:18:59,016 [p:206,w:0,m:0] [MainThread] At database version 141
tail: unrecognized file system type 0x794c7630 for ‘/home/galaxy/logs/uwsgi.log’. please report this to bug-coreutils@gnu.org. reverting to polling

Also, here's my 'docker info'

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 18.09.0
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: c4446665cb9c30056f4998ed953e6d4ff22c7c39
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-70-generic
Operating System: Ubuntu 16.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 20
Total Memory: 62.84GiB
Name: GalaxyHU-server
ID: 3ZS6:YDYV:MDBR:2LRA:ZTJ7:3YWR:7B2J:4IAR:G6QL:7754:4L6O:QZBK
Docker Root Dir: /media/galaxy/galaxy-docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

Thanks in advance for the help - please let me know what other info you need.

bgruening commented 5 years ago

@zslastman sorry missed that :(

Is that solved or can I still help you?

zslastman commented 5 years ago

HI Bjorn - no worries. I think that specific problem may have been solved, I followed the migration instructions and now have what looks like a running galaxy instance, except

a) One of my users is getting a ‘file not found’ error for a file that exists, when they try to run a tool - I can see it from the terminal inside the galaxy instance.

b) The ‘impersonate users’ button is now gone, making it difficult for me to debug the problem.

I”m trying to fix b) and I can only find old issues about impersonate users not working with external authentication. I thought maybe the new config file (which definitely has allow_impersonate_users=True) might not be getting read.

I’ve been kind of saddled with this instance despite not having much expertise, so apologies if it seems like I”m winging it….

Best Dermot

On 4. Jan 2019, at 11:54, Björn Grüning notifications@github.com wrote:

@zslastman https://github.com/zslastman sorry missed that :(

Is that solved or can I still help you?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bgruening/docker-galaxy-stable/issues/466#issuecomment-451413024, or mute the thread https://github.com/notifications/unsubscribe-auth/AFOyNrgN4RQO8_VirzCGIgGIgKVwpmCjks5u_zLjgaJpZM4ZYGyu.

bgruening commented 5 years ago

@zslastman for b) this option is moved to the users panel. Go to the admin view and then to Users pick your users and select from the Option impersonate. Hope that helps.

zslastman commented 5 years ago

Hi Bjorn, thanks for the reply.

That's where it was before alright, but it doesn't seem to be there now. There's no button for it anywhere on the users screen, and clicking on the options for a user gives me only 4 options:

image

bgruening commented 5 years ago

Following up in gitter ...