search

jazzdd86 / alpine-flask

72 stars 51 forks source link

I can't use Docker in my Flask app #2

Closed roipoussiere closed 6 years ago

roipoussiere commented 7 years ago

I ran the alpine-flask container:

docker run --name flaskapp -p 8083:80 -v /path/to/app/:/app -v /var/run/docker.sock:/var/run/docker.sock -d jazzdd/alpine-flask:python3 -o gid

... and then executed a docker info on it with docker exec -it flaskapp docker info, and it just works perfectly:

Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 12
[...]

But, when I execute this same command from my Flask app, it seems that the docker daemon is not running. Here is the app log:

fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/8) Installing libmnl (1.0.4-r0)
(2/8) Installing libnftnl-libs (1.0.7-r0)
(3/8) Installing iptables (1.6.0-r0)
(4/8) Installing xz (5.2.2-r1)
(5/8) Installing libseccomp (2.3.1-r0)
(6/8) Installing docker (1.12.6-r0)
Executing docker-1.12.6-r0.pre-install
(7/8) Installing linux-pam (1.2.1-r0)
(8/8) Installing shadow (4.2.1-r8)
Executing busybox-1.25.1-r0.trigger
OK: 191 MiB in 42 packages
groupmod: invalid group ID 'gid'
Adding user nginx to group docker
Running app in production mode!
2017/05/19 16:52:17 [notice] 15#15: using the "epoll" event method
2017/05/19 16:52:17 [notice] 15#15: nginx/1.10.3
2017/05/19 16:52:17 [notice] 15#15: OS: Linux 4.4.0-78-generic
2017/05/19 16:52:17 [notice] 15#15: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2017/05/19 16:52:17 [notice] 16#16: start worker processes
2017/05/19 16:52:17 [notice] 16#16: start worker process 18
2017/05/19 16:52:17 [notice] 16#16: start worker process 19
2017/05/19 16:52:17 [notice] 16#16: start worker process 20
2017/05/19 16:52:17 [notice] 16#16: start worker process 21
[uWSGI] getting INI configuration from /app.ini
*** Starting uWSGI 2.0.14 (64bit) on [Fri May 19 16:52:17 2017] ***
compiled with version: 6.2.1 20160822 on 22 February 2017 22:05:04
os: Linux-4.4.0-78-generic #99-Ubuntu SMP Thu Apr 27 15:29:09 UTC 2017
nodename: b4ea444303ed
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 12
current working directory: /app
writing pidfile to /run/.pid
detected binary path: /usr/sbin/uwsgi
setgid() to 101
set additional group 82 (www-data)
set additional group 103 (docker)
setuid() to 100
chdir() to /app
*** WARNING: you are running uWSGI without its master process manager ***
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/uwsgiApp.sock fd 3
Python version: 3.5.2 (default, Dec 22 2016, 10:15:38)  [GCC 6.2.1 20160822]
Python main interpreter initialized at 0x564065903000
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 332288 bytes (324 KB) for 8 cores
*** Operational MODE: preforking+threaded ***
WSGI app 0 (mountpoint='') ready in 2 seconds on interpreter 0x564065903000 pid: 17 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (pid: 17, cores: 2)
spawned uWSGI worker 2 (pid: 22, cores: 2)
spawned uWSGI worker 3 (pid: 23, cores: 2)
spawned uWSGI worker 4 (pid: 24, cores: 2)
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
[pid: 24|app: 0|req: 1/1] 172.17.0.1 () {40 vars in 520 bytes} [Fri May 19 16:52:21 2017] GET / => generated 2 bytes in 28 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
172.17.0.1 - - [19/May/2017:16:52:21 +0000] "GET / HTTP/1.1" 200 2 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0"

I think it's because I am logged as root with docker exec and as nginx on the Flask app... But I'm new to both Docker and Flask and I don't really know how to deal with it...

Note that I also tried with :latest but the containers stops immediatly with this error:

touch: /debug0: Permission denied
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
/entrypoint.sh: line 23: groupmod: command not found
/entrypoint.sh: line 24: gpasswd: command not found
Running app in production mode!
nginx: [alert] could not open error log file: open() "/var/lib/nginx/logs/error.log" failed (13: Permission denied)
2017/05/19 16:55:40 [warn] 11#11: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:1
2017/05/19 16:55:40 [emerg] 11#11: bind() to 0.0.0.0:80 failed (13: Permission denied)
nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)

And fyi here is my very simple app.py used for this test:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from flask import Flask
from subprocess import call

app = Flask(__name__)

@app.route('/', methods=['GET'])
def update():
    call(["docker", "info"])
    return 'ok'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=80, debug=True)

Is that a bug or just a newbie issue?

jazzdd86 commented 7 years ago

I think it is just a newbie issue ;-) See this line groupmod: invalid group ID 'gid' You have to actually define the gid used for the docker group. The default docker group id is 995. So your command should look like this:

docker run --name flaskapp -p 8083:80 -v /path/to/app/:/app -v /var/run/docker.sock:/var/run/docker.sock -d jazzdd/alpine-flask:python3 -o 995
roipoussiere commented 7 years ago

Ok, thank you!

Maybe you can be clearer about this on the readme, for example:

docker run --name flaskapp --restart=always \
   -p 80:80 \
   -v /path/to/app/:/app \
   -v /var/run/docker.sock:/var/run/docker.sock \
   -d jazzdd/alpine-flask -o <docker gid>

Where <docker gid> is ...

roipoussiere commented 7 years ago

Sorry, I am still confused here...

I tested:

docker run --name flaskapp -p 8002:80 -v /my/app/:/app -v /var/run/docker.sock:/var/run/docker.sock -d jazzdd/alpine-flask:python3 -o 995

And also:

docker run --name flaskapp -p 8002:80 -v /my/app/:/app -v /var/run/docker.sock:/var/run/docker.sock -d jazzdd/alpine-flask:python3 -o $(id -u $(whoami))

In both cases, the app log is roughly the same as above (exept for groupmod: invalid group ID 'gid'):

fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/8) Installing libmnl (1.0.4-r0)
(2/8) Installing libnftnl-libs (1.0.7-r0)
(3/8) Installing iptables (1.6.0-r0)
(4/8) Installing xz (5.2.2-r1)
(5/8) Installing libseccomp (2.3.1-r0)
(6/8) Installing docker (1.12.6-r0)
Executing docker-1.12.6-r0.pre-install
(7/8) Installing linux-pam (1.2.1-r0)
(8/8) Installing shadow (4.2.1-r8)
Executing busybox-1.25.1-r0.trigger
OK: 191 MiB in 42 packages
Adding user nginx to group docker
Running app in production mode!
2017/05/22 09:10:39 [notice] 15#15: using the "epoll" event method
2017/05/22 09:10:39 [notice] 15#15: nginx/1.10.3
2017/05/22 09:10:39 [notice] 15#15: OS: Linux 4.4.0-78-generic
2017/05/22 09:10:39 [notice] 15#15: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2017/05/22 09:10:39 [notice] 16#16: start worker processes
2017/05/22 09:10:39 [notice] 16#16: start worker process 18
2017/05/22 09:10:39 [notice] 16#16: start worker process 19
2017/05/22 09:10:39 [notice] 16#16: start worker process 20
2017/05/22 09:10:39 [notice] 16#16: start worker process 21
[uWSGI] getting INI configuration from /app.ini
*** Starting uWSGI 2.0.14 (64bit) on [Mon May 22 09:10:39 2017] ***
compiled with version: 6.2.1 20160822 on 22 February 2017 22:05:04
os: Linux-4.4.0-78-generic #99-Ubuntu SMP Thu Apr 27 15:29:09 UTC 2017
nodename: 9b86ebc7492a
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 12
current working directory: /app
writing pidfile to /run/.pid
detected binary path: /usr/sbin/uwsgi
setgid() to 101
set additional group 82 (www-data)
set additional group 1000 (docker)
setuid() to 100
chdir() to /app
*** WARNING: you are running uWSGI without its master process manager ***
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/uwsgiApp.sock fd 3
Python version: 3.5.2 (default, Dec 22 2016, 10:15:38)  [GCC 6.2.1 20160822]
Python main interpreter initialized at 0x56517eac9000
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 332288 bytes (324 KB) for 8 cores
*** Operational MODE: preforking+threaded ***
WSGI app 0 (mountpoint='') ready in 2 seconds on interpreter 0x56517eac9000 pid: 17 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (pid: 17, cores: 2)
spawned uWSGI worker 2 (pid: 22, cores: 2)
spawned uWSGI worker 3 (pid: 23, cores: 2)
spawned uWSGI worker 4 (pid: 24, cores: 2)
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
[pid: 24|app: 0|req: 1/1] 172.17.0.1 () {44 vars in 647 bytes} [Mon May 22 09:13:08 2017] POST /update/amdadb => generated 202 bytes in 31 msecs (HTTP/1.1 200) 2 headers in 80 bytes (1 switches on core 0)
172.17.0.1 - - [22/May/2017:09:13:08 +0000] "POST /update/amdadb HTTP/1.1" 200 202 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0"
jazzdd86 commented 7 years ago

Sorry my fault. Default docker group id is 996 and not 995. To see your docker GID just type:

user@workstation % id
uid=1000(***) gid=1000(***) Gruppen=1000(***),996(docker)

$(id -u $(whoami)) won't work as it outputs the user ID of your current user and not the group ID.

I just tested your little script with GID=996 and works fine.

roipoussiere commented 7 years ago

Arf, not better.

In my case the docker group id is 999:

id                                                                                                                                                                         
uid=1000(nathanael) gid=1000(nathanael) groupes=1000(nathanael),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),113(lpadmin),129(sambashare),999(docker)

And Alpine says that GID '999' already exists. :/

docker run --name flaskapp -p 8002:80 -v /home/nathanael/VO_DaCHS/hook_handler/app/:/app -v /var/run/docker.sock:/var/run/docker.sock -d jazzdd/alpine-flask:python3 -o 999
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/8) Installing libmnl (1.0.4-r0)
(2/8) Installing libnftnl-libs (1.0.7-r0)
(3/8) Installing iptables (1.6.0-r0)
(4/8) Installing xz (5.2.2-r1)
(5/8) Installing libseccomp (2.3.1-r0)
(6/8) Installing docker (1.12.6-r0)
Executing docker-1.12.6-r0.pre-install
(7/8) Installing linux-pam (1.2.1-r0)
(8/8) Installing shadow (4.2.1-r8)
Executing busybox-1.25.1-r0.trigger
OK: 191 MiB in 42 packages
groupmod: GID '999' already exists
Adding user nginx to group docker
Running app in production mode!
2017/05/23 08:02:48 [notice] 15#15: using the "epoll" event method
2017/05/23 08:02:48 [notice] 15#15: nginx/1.10.3
2017/05/23 08:02:48 [notice] 15#15: OS: Linux 4.4.0-78-generic
2017/05/23 08:02:48 [notice] 15#15: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2017/05/23 08:02:48 [notice] 16#16: start worker processes
2017/05/23 08:02:48 [notice] 16#16: start worker process 18
2017/05/23 08:02:48 [notice] 16#16: start worker process 19
2017/05/23 08:02:48 [notice] 16#16: start worker process 20
2017/05/23 08:02:48 [notice] 16#16: start worker process 21
[uWSGI] getting INI configuration from /app.ini
*** Starting uWSGI 2.0.14 (64bit) on [Tue May 23 08:02:48 2017] ***
compiled with version: 6.2.1 20160822 on 22 February 2017 22:05:04
os: Linux-4.4.0-78-generic #99-Ubuntu SMP Thu Apr 27 15:29:09 UTC 2017
nodename: c96ec8120533
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 12
current working directory: /app
writing pidfile to /run/.pid
detected binary path: /usr/sbin/uwsgi
setgid() to 101
set additional group 82 (www-data)
set additional group 103 (docker)
setuid() to 100
chdir() to /app
*** WARNING: you are running uWSGI without its master process manager ***
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /run/uwsgiApp.sock fd 3
Python version: 3.5.2 (default, Dec 22 2016, 10:15:38)  [GCC 6.2.1 20160822]
Python main interpreter initialized at 0x557224a21000
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 332288 bytes (324 KB) for 8 cores
*** Operational MODE: preforking+threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x557224a21000 pid: 17 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (pid: 17, cores: 2)
spawned uWSGI worker 2 (pid: 22, cores: 2)
spawned uWSGI worker 3 (pid: 23, cores: 2)
spawned uWSGI worker 4 (pid: 24, cores: 2)
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
[pid: 24|app: 0|req: 1/1] 172.17.0.1 () {44 vars in 647 bytes} [Tue May 23 08:05:42 2017] POST /update/amdadb => generated 202 bytes in 31 msecs (HTTP/1.1 200) 2 headers in 80 bytes (1 switches on core 0)
172.17.0.1 - - [23/May/2017:08:05:42 +0000] "POST /update/amdadb HTTP/1.1" 200 202 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0"
jazzdd86 commented 7 years ago

Yes, alpine has a group called ping with id 999. I . Why is your docker running at group 999?

You could do a workaround. Use 996 as GID in your docker command and use following command at your running container: docker exec -it flaskapp gpasswd -a nginx ping

It appends user nginx to group ping, which has ID 999. This will work as if your docker group would have 999 as ID. Hope this works.