pschiffe / docker-pdns

Docker images for PowerDNS
MIT License
271 stars 80 forks source link

nginx error with pdns-admin-uwsgi and pdns-admin-static #2

Closed boarder981 closed 7 years ago

boarder981 commented 7 years ago

I have a server running Docker version 17.04.0-ce that has pdns-admin running in a container. I used the pschiffe/docker-pdns image, which works fine when launched as follows:

docker run -dt -P --name pdns-admin \
  --privileged \
  -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
  -e PDNS_ADMIN_SQLA_DB_HOST="'10.a.b.c'" \
  -e PDNS_ADMIN_SQLA_DB_PORT="'3306'" \
  -e PDNS_ADMIN_SQLA_DB_USER="'pdnsdbuser'" \
  -e PDNS_ADMIN_SQLA_DB_PASSWORD="'mysqlsupersecret'" \
  -e PDNS_ADMIN_SQLA_DB_NAME="'pdnsadmin'" \
  -e PDNS_ADMIN_PDNS_STATS_URL="'http://10.x.y.z:8081/'" \
  -e PDNS_ADMIN_PDNS_API_KEY="'somelongkey'" \
  -e PDNS_ADMIN_PDNS_VERSION="'4.0.0'" \
  -e PDNS_ADMIN_LDAP_TYPE="'ldap'" \
  -e PDNS_ADMIN_LDAP_URI="'ldaps://ldap.mycompany.com:636'" \
  -e PDNS_ADMIN_LDAP_USERNAME="'uid=pdnsuser@mycompany.com,ou=people,dc=mycompany,dc=com'" \
  -e PDNS_ADMIN_LDAP_PASSWORD="'ldapsecretpass'" \
  -e PDNS_ADMIN_LDAP_SEARCH_BASE="'ou=People,dc=mycompany,dc=com'" \
  -e PDNS_ADMIN_LDAP_USERNAMEFIELD="'uid'" \
  -e PDNS_ADMIN_LDAP_FILTER="'(accessRole=uniqueIdentifier=pdnsadmin,ou=roles)'" \
  pschiffe/pdns-admin

I have a separate Docker server, which has been upgraded to version 17.09.0-ce. When the pdns-admin container is launched with the same parameters above, it doesn't work properly. Here is log output when the container is started

systemd 233 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN default-hierarchy=hybrid)
Detected virtualization docker.
Detected architecture x86-64.

Welcome to Fedora 26 (Twenty Six)!

Set hostname to <8e19df351cbb>.
Initializing machine ID from random generator.
[ INFO ] Unnecessary job for /dev/sda1 was removed.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Reached target Swap.
[  OK  ] Set up automount Arbitrary Executab…rmats File System Automount Point.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
[  OK  ] Listening on Process Core Dump Socket.
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Encrypted Volumes.
[  OK  ] Reached target Paths.
[  OK  ] Created slice System Slice.
         Mounting Configuration File System...
         Starting Journal Service...
         Starting Apply Kernel Variables...
         Starting Create System Users...
[  OK  ] Reached target Slices.
         Mounting Debug File System...
tmp.mount: Directory /tmp to mount over is not empty, mounting anyway.
         Mounting Temporary Directory...
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Create System Users.
[  OK  ] Mounted Configuration File System.
[  OK  ] Mounted Debug File System.
[  OK  ] Mounted Temporary Directory.
[  OK  ] Reached target Local File Systems.
         Starting Rebuild Journal Catalog...
         Starting Rebuild Dynamic Linker Cache...
[426335.069289] [systemd-sysctl  OK  [25]: ] Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directoryStarted Journal Service.

[426335.069435] systemd-sysctl[25]: Couldn't write 'fq_codel' to 'net/core/default_qdisc', ignoring: No such file or directory
         Starting Flush Journal to Persistent Storage...
[426335.069869] systemd[1]: Starting Flush Journal to Persistent Storage...

When I browse to http://hostip:port, I get the big "nginx error" page. Also, I don't see anything streaming in the container logs when I try to hit it.

Since I see that the pdns-admin image is now deprecated, I tried to use the newer pdns-admin-uwsgi and pdns-admin-static images. This is how they were launched...

Started pdns-admin-uwsgi

docker run -dt --name pdns-admin-uwsgi \
  --privileged \
  -v pdns-admin-upload:/opt/powerdns-admin/upload \
  -e PDNS_ADMIN_SQLA_DB_HOST="'10.x.x.x'" \
  -e PDNS_ADMIN_SQLA_DB_PORT="'3306'" \
  -e PDNS_ADMIN_SQLA_DB_USER="'pdnsdbuser'" \
  -e PDNS_ADMIN_SQLA_DB_PASSWORD="'mysqlsupersecret'" \
  -e PDNS_ADMIN_SQLA_DB_NAME="'pdnsadmin'" \
  -e PDNS_ADMIN_PDNS_STATS_URL="'http://10.x.x.x:8081/'" \
  -e PDNS_ADMIN_PDNS_API_KEY="'somelongkey'" \
  -e PDNS_ADMIN_PDNS_VERSION="'4.0.0'" \
  -e PDNS_ADMIN_LDAP_TYPE="'ldap'" \
  -e PDNS_ADMIN_LDAP_URI="'ldaps://ldap.mycompany.com:636'" \
  -e PDNS_ADMIN_LDAP_USERNAME="'uid=pdnsuser@mycompany.com,ou=people,dc=mycompany,dc=com'" \
  -e PDNS_ADMIN_LDAP_PASSWORD="'ldapsecretpass'" \
  -e PDNS_ADMIN_LDAP_SEARCH_BASE="'ou=People,dc=mycompany,dc=com'" \
  -e PDNS_ADMIN_LDAP_USERNAMEFIELD="'uid'" \
  -e PDNS_ADMIN_LDAP_FILTER="'(accessRole=uniqueIdentifier=pdnsadmin,ou=roles)'" \
  pschiffe/pdns-admin-uwsgi

Log output:

[uWSGI] getting INI configuration from /etc/uwsgi.ini
setting capability setgid [6]
setting capability setuid [7]
*** Starting uWSGI 2.0.15 (64bit) on [Mon Oct  9 19:43:05 2017] ***
compiled with version: 7.1.1 20170503 (Red Hat 7.1.1-1) on 19 May 2017 15:43:52
os: Linux-3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016
nodename: 9c7a727ecc74
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /opt/powerdns-admin
writing pidfile to /run/uwsgi/uwsgi.pid
detected binary path: /usr/sbin/uwsgi
setgid() to 996
setuid() to 997
your memory page size is 4096 bytes
detected max file descriptor number: 65536
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
your mercy for graceful operations on workers is 60 seconds
*** Operational MODE: no-workers ***
spawned uWSGI master process (pid: 1)
*** Stats server enabled on /run/uwsgi/stats.sock fd: 6 ***
*** starting uWSGI Emperor ***
[emperor-tyrant] dropping privileges to 997 996 for instance pdns-admin.ini
*** has_emperor mode detected (fd: 6) ***
[uWSGI] getting INI configuration from pdns-admin.ini
*** Starting uWSGI 2.0.15 (64bit) on [Mon Oct  9 19:43:05 2017] ***
compiled with version: 7.1.1 20170503 (Red Hat 7.1.1-1) on 19 May 2017 15:43:52
os: Linux-3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016
nodename: 9c7a727ecc74
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /etc/uwsgi.d
writing pidfile to /run/uwsgi/pdns-admin.pid
detected binary path: /usr/sbin/uwsgi
chdir() to /opt/powerdns-admin
your memory page size is 4096 bytes
detected max file descriptor number: 65536
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:9494 fd 3
Python version: 2.7.13 (default, Sep  5 2017, 08:53:59)  [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
Python main interpreter initialized at 0x11abc40
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 161920 bytes (158 KB) for 1 cores
*** Operational MODE: single process ***
added /opt/powerdns-admin/ to pythonpath.
mounting run.py on /
WSGI app 0 (mountpoint='/') ready in 2 seconds on interpreter 0x11abc40 pid: 18 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 18)
Mon Oct  9 19:43:08 2017 - [emperor] vassal pdns-admin.ini has been spawned
spawned uWSGI worker 1 (pid: 20, cores: 1)
Mon Oct  9 19:43:08 2017 - [emperor] vassal pdns-admin.ini is ready to accept requests

...and then started pdns-admin-static...

docker run -dt -p 8999:80 --name pdns-admin-static \
  --link pdns-admin-uwsgi:pdns-admin-uwsgi \
  pschiffe/pdns-admin-static

I get a 502 when trying to hit it in my browser. Log output...

2017/10/09 19:46:33 [error] 7#7: *1 upstream prematurely closed connection while reading response header from upstream, client: 10.x.x.x, server: localhost, request: "GET / HTTP/1.1", upstream: "uwsgi://172.17.0.7:9494", host: "dockerhost.mycompany.com:8999"
10.x.x.x - - [09/Oct/2017:19:46:33 +0000] "GET / HTTP/1.1" 502 537 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.101 Safari/537.36" "-"
2017/10/09 19:46:33 [error] 7#7: *1 upstream prematurely closed connection while reading response header from upstream, client: 10.x.x.x, server: localhost, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://172.17.0.7:9494", host: "dockerhost.mycompany.com:8999", referrer: "http://dockerhost.mycompany.com:8999/"
10.x.x.x - - [09/Oct/2017:19:46:33 +0000] "GET /favicon.ico HTTP/1.1" 502 537 "http://dockerhost.mycompany.com:8999/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.101 Safari/537.36" "-"
2017/10/09 19:46:45 [error] 7#7: *1 upstream prematurely closed connection while reading response header from upstream, client: 10.29.23.25, server: localhost, request: "GET / HTTP/1.1", upstream: "uwsgi://172.17.0.7:9494", host: "dockerhost.mycompany.com:8999"
10.x.x.x - - [09/Oct/2017:19:46:45 +0000] "GET / HTTP/1.1" 502 537 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.101 Safari/537.36" "-"
2017/10/09 19:46:45 [error] 7#7: *1 upstream prematurely closed connection while reading response header from upstream, client: 10.29.23.25, server: localhost, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://172.17.0.7:9494", host: "dockerhost.mycompany.com:8999", referrer: "http://dockerhost.mycompany.com:8999/"
10.x.x.x - - [09/Oct/2017:19:46:45 +0000] "GET /favicon.ico HTTP/1.1" 502 537 "http://dockerhost.mycompany.com:8999/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.101 Safari/537.36" "-"

I have no idea what I'm doing wrong. Have tried a few different things, but can't quite figure this out. Any ideas or solutions would be much appreciated!

pschiffe commented 7 years ago

Hello, try to post the uwsgi container logs, after you see the error in the static container.

boarder981 commented 7 years ago

Thank you for the quick response! Here are the pdns-admin-uwsgi container logs

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/werkzeug/contrib/fixers.py", line 152, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/powerdns-admin/app/views.py", line 1203, in home
    pages = Page.query.all()
  File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2423, in all
    return list(self)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2571, in __iter__
    return self._execute_and_instances(context)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2586, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
    raise errorvalue
sqlalchemy.exc.ProgrammingError: (_mysql_exceptions.ProgrammingError) (1146, "Table 'powerdnsadmin.page' doesn't exist") [SQL: u'SELECT page.id AS page_id, page.name AS page_name, page.title AS page_title, page.url AS page_url, page.icon AS page_icon, page.roles AS page_roles, page.published AS page_published, page.content AS page_content \nFROM page']
[pid: 21|app: 0|req: 1/1] 10.x.x.x () {40 vars in 707 bytes} [Tue Oct 10 14:08:49 2017] GET / => generated 0 bytes in 204 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)
announcing my loyalty to the Emperor...
Tue Oct 10 14:08:50 2017 - [emperor] vassal pdns-admin.ini is now loyal
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/werkzeug/contrib/fixers.py", line 152, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/powerdns-admin/app/views.py", line 382, in custompage
    login()
  File "/opt/powerdns-admin/app/views.py", line 42, in login
    pages = Page.query.all()
  File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2423, in all
    return list(self)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2571, in __iter__
    return self._execute_and_instances(context)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2586, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
    raise errorvalue
sqlalchemy.exc.ProgrammingError: (_mysql_exceptions.ProgrammingError) (1146, "Table 'powerdnsadmin.page' doesn't exist") [SQL: u'SELECT page.id AS page_id, page.name AS page_name, page.title AS page_title, page.url AS page_url, page.icon AS page_icon, page.roles AS page_roles, page.published AS page_published, page.content AS page_content \nFROM page']
[pid: 21|app: 0|req: 2/2] 10.x.x.x () {42 vars in 718 bytes} [Tue Oct 10 14:08:50 2017] GET /favicon.ico => generated 0 bytes in 9 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)

So it's looking for the page table, which doesn't exist in the DB we've been using with the old docker-pdns. Here are the tables that exist currently

mysql> show tables;
+-------------------------------+
| Tables_in_powerdnsadmin |
+-------------------------------+
| domain                        |
| domain_setting                |
| domain_user                   |
| history                       |
| migrate_version               |
| role                          |
| setting                       |
| user                          |
+-------------------------------+
8 rows in set (0.00 sec)

I now see that your links to the git repos for the old pdns-admin and new pdns-admin-uwsgi are different. They look eerily similar, but I guess the latter uses a different database format.

If I create a new database as outlined here and update the container environment variables accordingly, will it automatically create the necessary tables when I launch the pdns-admin-uwsgi container?

pschiffe commented 7 years ago

You don't even need to create new db, it will be created automatically, see https://github.com/pschiffe/docker-pdns/blob/master/pdns-admin-uwsgi/docker-cmd.sh#L40

Just update the env vars, and you should be good to go.

boarder981 commented 7 years ago

Thank you! I now have the 0x97-Admin GUI running with a different database.

However, now users need to be re-created, history is blank, and settings are default (of course, since it's a fresh DB). I noticed there are some scripts in the container db_migrate.py and db_upgrade.py. In order to retain our prior configuration, is there a way to upgrade the original database we were using with the old pdns-admin image?

pschiffe commented 7 years ago

I'm sorry, I've never tried that. Maybe even copying the content of some tables from old to new could work (like users and history..)