search

unbit / uwsgi

uWSGI application server container
http://projects.unbit.it/uwsgi
Other
3.45k stars 687 forks source link

Segmentation Fault with `uid` option on OpenBSD 6.4 #1901

Open iredmail opened 5 years ago

iredmail commented 5 years ago

The issue is uwsgi always gives Segmentation Fault error while i have uid option in ini file or command line, but it works if i switch to user with doas (similar to sudo) and run command.

NOTE: Same python code, same ini config file works fine on OpenBSD 6.3 (amd64) with uwsgi-2.0.17.

Is it bug of uwsgi? or my ini file, python code, or the OS (OpenBSD 6.4)? How can i get uid option working?

Simple web.py (0.38) application: /opt/main.py, owned by user mlmmj:mlmmj, permission 0755:

import web

urls = (
    '/', 'index'
)

class index:
    def GET(self):
        return "Hello, world!"

app = web.application(urls, globals())
application = app.wsgifunc()

Config file /opt/openbsd.ini:

[uwsgi]
master = true
vhost = true
enable-threads = true
processes = 5

http-socket = 127.0.0.1:7790
chdir = /opt
wsgi-file = main.py
uid = mlmmj
gid = mlmmj

Run uwsgi as root (it should switch to mlmmj:mlmmj user with the uid = and gid = setting in ini file):

# uwsgi --ini /opt/openbsd.ini
[uWSGI] getting INI configuration from openbsd.ini
*** Starting uWSGI 2.1-dev-49ac7bd8 (64bit) on [Fri Oct 19 00:17:16 2018] ***
compiled with version: 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final) on 18 October 2018 20:54:36
os: OpenBSD-6.4 GENERIC#349
nodename: ob64.localdomain
machine: amd64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /opt
detected binary path: uwsgi
dropping root privileges as early as possible
setgid() to 2003
setuid() to 2003
chdir() to /opt
your processes number limit is 1310
your memory page size is 4096 bytes
detected max file descriptor number: 128
VirtualHosting mode enabled.
lock engine: ipcsem
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 127.0.0.1:7790 fd 3
dropping root privileges after socket binding
Python version: 2.7.15 (default, Oct 11 2018, 12:36:27)  [GCC 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final)]
Python main interpreter initialized at 0x17962aeff400
dropping root privileges after plugin initialization
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
your request buffer size is 4096 bytes
mapped 437712 bytes (427 KB) for 5 cores
*** Operational MODE: preforking ***
!!! uWSGI process 69114 got Segmentation Fault !!!

Comment out uid = and gid =, then it works, but it's running as root user, this is definitely not what i want.

I switch to mlmmj user then run uwsgi, it works, and runs as mlmmj user/group:

# doas -u mlmmj /usr/local/bin/uwsgi --ini openbsd.ini
[uWSGI] getting INI configuration from openbsd.ini
*** Starting uWSGI 2.1-dev-49ac7bd8 (64bit) on [Fri Oct 19 00:19:28 2018] ***
compiled with version: 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final) on 18 October 2018 20:54:36
os: OpenBSD-6.4 GENERIC#349
nodename: ob64.localdomain
machine: amd64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /opt
detected binary path: /usr/local/bin/uwsgi
dropping root privileges as early as possible
chdir() to /opt
your processes number limit is 128
your memory page size is 4096 bytes
detected max file descriptor number: 512
VirtualHosting mode enabled.
lock engine: ipcsem
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 127.0.0.1:7790 fd 3
dropping root privileges after socket binding
Python version: 2.7.15 (default, Oct 11 2018, 12:36:27)  [GCC 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final)]
Python main interpreter initialized at 0x8af8313db00
dropping root privileges after plugin initialization
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
your request buffer size is 4096 bytes
mapped 437712 bytes (427 KB) for 5 cores
*** Operational MODE: preforking ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x8af8313db00 pid: 30611 (default app)
dropping root privileges after application loading
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 30611)
spawned uWSGI worker 1 (pid: 19071, cores: 1)
spawned uWSGI worker 2 (pid: 618, cores: 1)
spawned uWSGI worker 3 (pid: 92489, cores: 1)
spawned uWSGI worker 4 (pid: 84733, cores: 1)
spawned uWSGI worker 5 (pid: 22707, cores: 1)
asiekierka commented 5 years ago

Can confirm both the issue and workaround.

raratiru commented 5 years ago

Same here. Upgrading to OpenBSD-6.4 either by doing the upgrade process or by reinstalling the same configuration fies from scratch with ansible I get Segmentation Fault.

My config:

# cat /etc/uwsgi.ini
[uwsgi]
chdir = /home/user/django_project
virtualenv = /home/user/.virtualenvs/environment
module = django_project.wsgi:application
env = DJANGO_SETTINGS_MODULE=django_project.settings.settingsp
master = true
need-app = true
uid = 67
gid = 67
daemonize2 = /var/log/uwsgi/daemon.log
safe-pidfile2 = /var/run/uwsgi/uwsgi.pid
ftok = /etc/rc.d/uwsgi
fastcgi-socket = /var/www/run/uwsgi.socket
chown-socket = www:www
processes = 8
cheaper = 1
cheaper-initial = 3
cheaper-step = 1
log-format =  %(status) | %(uri) | %(method) | %(referer) | %(time) | %(uagent)
req-logger = file:/var/log/uwsgi/request.log
logger = file:/var/log/uwsgi/main.log
vacuum = true
harakiri = 20
no-orphans = True
reload-mercy = 8
max-requests = 5000
reload-on-rss = 104

The diff between the manually run command on both (working) OpenBSD-6.3 and (not working) OpenBSD-6.4 are the following:

# /home/user/.virtualenvs/environment/bin/uwsgi --ini 
-*** Starting uWSGI 2.0.17.1 (64bit) on [Mon Nov  5 19:13:38 2018] ***
-compiled with version: 4.2.1 Compatible OpenBSD Clang 5.0.1 (tags/RELEASE_501/final) on 30 July 2018 00:53:43
-os: OpenBSD-6.3 GENERIC#11
+*** Starting uWSGI 2.0.17.1 (64bit) on [Mon Nov  5 19:11:44 2018] ***
+compiled with version: 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final) on 05 November 2018 15:57:59
+os: OpenBSD-6.4 GENERIC#349

-Python version: 3.6.4 (default, Mar 27 2018, 09:58:38)  [GCC 4.2.1 Compatible OpenBSD Clang 5.0.1 (tags/RELEASE_501/final)]
+Python version: 3.6.6 (default, Oct 11 2018, 12:39:19)  [GCC 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final)]

-Python main interpreter initialized at 0x975989d0720
+Python main interpreter initialized at 0x1a874fade720

-WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x975989d0720 pid: 96831 (default app)
-*** daemonizing uWSGI ***
+!!! uWSGI process 71555 got Segmentation Fault !!!
+unlink(): No such file or directory [core/uwsgi.c line 1638]
+unlink(): Permission denied [core/uwsgi.c line 1659]
iredmail commented 5 years ago

Since i cannot wait for uwsgi team to "fix" it, i have to run uwsgi app as a standalone service:

Here's my rc script if you need a sample. For more details, just read OpenBSD manual pages:

#!/bin/sh

RUN_DIR='/var/run/myapp'
PID_FILE="${RUN_DIR}/myapp.pid"
UWSGI_INI_FILE='/opt/www/myapp/rc_scripts/uwsgi/openbsd.ini'

daemon="/usr/local/bin/uwsgi --ini ${UWSGI_INI_FILE} --log-syslog --pidfile ${PID_FILE} --daemonize /dev/null"
daemon_user='myapp'
daemon_group='myapp'

. /etc/rc.d/rc.subr

rc_pre() {
    install -d -o ${daemon_user} -g ${daemon_group} -m 0775 ${RUN_DIR}
}

rc_stop() {
    kill -INT `cat ${PID_FILE}`
}

rc_cmd $1
raratiru commented 5 years ago

@iredmail Yup, that did it, thank you!

I will have to make some arrangements though because I am using fastcgi-socket which requires some preparations that need root permissions before dropping to the user that uwsgi will operate as.

paulc commented 5 years ago

I had the same problem - a bit more detail on the location of the crash below.

Looking at the location of the crash (plugins/python/python_plugin.c:876) I couldn't work out anything obvious that was causing this.


# uname -a
OpenBSD xxxx 6.4 GENERIC.MP#364 amd64
# id
uid=0(root) gid=0(wheel) groups=0(wheel), 2(kmem), 3(sys), 4(tty), 5(operator), 20(staff), 31(guest)
# uwsgi --version                                                                                                                                                                                                                                       
2.0.17.1
# gdb ./uwsgi 
GNU gdb (GDB) 7.12.1
...
Reading symbols from ./uwsgi...done.
>>> set args --fastcgi-socket /tmp/xxxx --wsgi-file app.py --uid www --gid www  
>>> run
─── Output/messages 
*** Starting uWSGI 2.1-dev-43b465ce (64bit) on [Sat Dec 22 23:13:08 2018] ***
compiled with version: 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final) on 22 December 2018 13:52:21
os: OpenBSD-6.4 GENERIC.MP#364
nodename: xxxx
machine: amd64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /home/paulc/uwsgi
detected binary path: /home/paulc/uwsgi/uwsgi
dropping root privileges as early as possible
setgid() to 67
setuid() to 67
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 1310
your memory page size is 4096 bytes
detected max file descriptor number: 128
lock engine: ipcsem
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/xxxx fd 3
dropping root privileges after socket binding
Python version: 3.6.6 (default, Oct 11 2018, 12:39:19)  [GCC 4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0xcaaa30da720
dropping root privileges after plugin initialization
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
your request buffer size is 4096 bytes
mapped 72952 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***

Program received signal SIGSEGV, Segmentation fault.
─── Assembly 
0x00000ca7d517e9b3 init_uwsgi_embedded_module+1587 mov    0xef0(%r14,%rbx,8),%rax
0x00000ca7d517e9bb init_uwsgi_embedded_module+1595 test   %rax,%rax
0x00000ca7d517e9be init_uwsgi_embedded_module+1598 je     0xca7d517e9f0 <init_uwsgi_embedded_module+1648>
0x00000ca7d517e9c0 init_uwsgi_embedded_module+1600 cmpb   $0x0,(%rax)
0x00000ca7d517e9c3 init_uwsgi_embedded_module+1603 je     0xca7d517e9f0 <init_uwsgi_embedded_module+1648>
0x00000ca7d517e9c5 init_uwsgi_embedded_module+1605 mov    $0x1,%esi
0x00000ca7d517e9ca init_uwsgi_embedded_module+1610 mov    %r13,%rdi
─── Registers 
   rax 0x00000caac179906c      rbx 0x0000000000000055      rcx 0x00000ca9de8fdb88      rdx 0x0000000000000000      rsi 0x0000000000000000      rdi 0x0000000000000000      rbp 0x00007f7ffffdaaf0      rsp 0x00007f7ffffdaaa0       r8 0x00000ca9de8fdb58  
    r9 0x00000ca9de8fdb80      r10 0x0000000000000007      r11 0x7cf52be363782a2f      r12 0x00000ca9de8feea0      r13 0x00007f7ffffdaab7      r14 0x00000ca7d545c1a0      r15 0x00000caa19b28878      rip 0x00000ca7d517e9c0   eflags [ PF IF RF ]        
    cs 0x0000002b               ss 0x00000023               ds 0x00000023               es 0x00000023               fs 0x00000023               gs 0x00000023          
─── Source 
871     uint8_t mtk;
872     for (i = 0; i <= 0xff; i++) {
873         // a bit of magic :P
874         mtk = i;
875                 if (uwsgi.magic_table[i]) {
876             if (uwsgi.magic_table[i][0] != 0) {
877                 PyDict_SetItem(py_magic_table, PyString_FromStringAndSize((char *) &mtk, 1), PyString_FromString(uwsgi.magic_table[i]));
878             }
879         }
880         }
881 
─── Stack 
[0] from 0x00000ca7d517e9c0 in init_uwsgi_embedded_module+1600 at plugins/python/python_plugin.c:876
(no arguments)
[1] from 0x00000ca7d517f1e8 in uwsgi_python_preinit_apps+184 at plugins/python/python_plugin.c:1132
(no arguments)
[+]
─── Threads 
[1] id 417464 from 0x00000ca7d517e9c0 in init_uwsgi_embedded_module+1600 at plugins/python/python_plugin.c:876
──────────────────────────────────────────────
init_uwsgi_embedded_module () at plugins/python/python_plugin.c:876
876                             if (uwsgi.magic_table[i][0] != 0) {
>>> bt
#0  init_uwsgi_embedded_module () at plugins/python/python_plugin.c:876
#1  0x00001d7e2ed7f1e8 in uwsgi_python_preinit_apps () at plugins/python/python_plugin.c:1132
#2  0x00001d7e2ed666b3 in uwsgi_start (v_argv=<optimized out>) at core/uwsgi.c:3227
#3  0x00001d7e2ed64a27 in uwsgi_setup (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at core/uwsgi.c:2644
#4  0x00001d7e2ed627d9 in main (argc=5, argv=0x5, envp=0x0) at core/uwsgi.c:2114
>>>
vigilancer commented 5 years ago

Same here

As workaround when using with rc.d change {rcexec}:

#/etc/rc.d/uwsgi

#!/bin/sh
#
daemon="/usr/local/bin/uwsgi"
. /etc/rc.d/rc.subr
rcexec="su - {YOUR_USER_HERE} -c"

rc_stop() {
     ${daemon} --stop /var/run/uwsgi.pid
}
rc_cmd $1

Also be sure to change logfile location if {YOUR_USER_HERE} does not have permissions to write to default location /var/log/.

$ uname -a
OpenBSD xxx 6.4 GENERIC#7 amd64
$ uwsgi --version
2.0.18
ercwyne commented 5 years ago

This will be fallout from uwsgi assuming the static content of getpwuid() will remain available. Check out the August 2018 OpenBSD CVS log entry for getpwent.c

https://cvsweb.openbsd.org/src/lib/libc/gen/getpwent.c

Drop privileges, access the 'static' buffer, and fail.

core/uwsgi.c should be using getpwuid_r()

--- uwsgi/core/uwsgi.c  Tue Jun 11 17:37:18 2019
+++ uwsgi.working/core/uwsgi.c  Wed Jun 12 09:33:37 2019
@@ -2302,8 +2302,16 @@
        uwsgi.magic_table['k'] = uwsgi_num2str(uwsgi.cpus);
        uwsgi.magic_table['['] = "\033";
        uwsgi.magic_table['u'] = uwsgi_num2str((int)getuid());
-       struct passwd *pw = getpwuid(getuid());
-       uwsgi.magic_table['U'] = pw ? pw->pw_name : uwsgi.magic_table['u'];
+
+       int pw_size = sysconf(_SC_GETPW_R_SIZE_MAX);
+       struct passwd pw, *pw_p;
+       char pw_buf[pw_size];
+
+       if(!getpwuid_r(getuid(), &pw, pw_buf, sizeof(pw_buf), &pw_p)) {
+               uwsgi.magic_table['U'] = pw.pw_name;
+       } else {
+               uwsgi.magic_table['U'] = uwsgi.magic_table['u'];
+       }
        uwsgi.magic_table['g'] = uwsgi_num2str((int)getgid());
        struct group *gr = getgrgid(getgid());
        uwsgi.magic_table['G'] = gr ? gr->gr_name : uwsgi.magic_table['g'];
ercwyne commented 5 years ago

I should mention that my patch is proof of concept only.

Are there any platforms that don't have _r functions?