search

unbit / uwsgi

uWSGI application server container
http://projects.unbit.it/uwsgi
Other
3.46k stars 691 forks source link

Segfault on latest Debian9 #1759

Open analytik opened 6 years ago

analytik commented 6 years ago

While running both uwsgi 2.0.15 and 2.0.17 with something like uwsgi --chdir=/project --http-socket 0.0.0.0:80 --module=app:api --pidfile=/tmp/uwsgi.pid --stats 127.0.0.1:1717 --stats-http --master --processes=2 --threads=2 --post-buffering --harakiri=30 --max-requests=5000 --vacuum --buffer-size=65535 --disable-logging --honour-stdin --py-autoreload=1

*** Starting uWSGI 2.0.17 (64bit) on [Wed Mar 14 10:18:27 2018] ***
compiled with version: 6.3.0 20170516 on 14 March 2018 10:17:26
os: Linux-4.14.19-coreos #1 SMP Wed Feb 14 04:08:06 UTC 2018
nodename: someproject-j36zv
machine: x86_64
clock source: unix
detected number of CPU cores: 2
current working directory: /project
writing pidfile to /tmp/uwsgi.pid
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
chdir() to /project
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:80 fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.6.4+ (default, Feb 12 2018, 08:25:03)  [GCC 7.3.0]
Python main interpreter initialized at 0x55c921281890
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 618762 bytes (604 KB) for 4 cores
*** Operational MODE: preforking+threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x55c921281890 pid: 7 (default app)
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 7)
spawned uWSGI worker 1 (pid: 12, cores: 2)
spawned uWSGI worker 2 (pid: 13, cores: 2)
*** Stats server enabled on 127.0.0.1:1717 fd: 13 ***
Python auto-reloader enabled
!!! uWSGI process 12 got Segmentation Fault !!!
*** backtrace of 12 ***
uwsgi(uwsgi_backtrace+0x35) [0x55c920724d85]
uwsgi(uwsgi_segfault+0x23) [0x55c920725133]
/lib/x86_64-linux-gnu/libc.so.6(+0x34f00) [0x7f537672df00]
/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0(PyBytes_AsString+0) [0x7f5376d79fd0]
uwsgi(uwsgi_python_autoreloader_thread+0x166) [0x55c920736be6]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x75aa) [0x7f53784805aa]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f53767efcbf]
*** end of backtrace ***

and then the segfault just keeps repeating every second or so.

The build itself seems to have been OK:

  Running setup.py bdist_wheel for uwsgi: started
  Running setup.py bdist_wheel for uwsgi: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/94/c9/63/e7aef2e745bb1231490847ee3785e3d0b5f274e1f1653f89c5

This started happening after rebuilding from the latest debian:stretch image. This wasn't happening a month ago.

Dockerfile for Debian:

FROM debian:stretch

ENV LC_ALL C.UTF-8
ENV TERM vt220

RUN apt-get update && apt-get install -y \
    curl \
    git \
    less \
    mercurial \
    nano \
    netcat \
    tcpdump \
    vim && \
    apt-get clean

Dockerfile for Python:

# Install dependencies
RUN echo "deb http://ftp.de.debian.org/debian unstable main" >> \
    etc/apt/sources.list && \
    apt-get update && apt-get install -y \
    python3.6 \
    python3.6-dev && \
    apt-get clean && \
    sed -i "/unstable/d" /etc/apt/sources.list

# Install pip
RUN curl https://bootstrap.pypa.io/get-pip.py | python3

Dockerfile for the project:

RUN apt-get update && apt-get install -y \
    gcc \
    libffi-dev \
    libpq-dev \
    libssl-dev && \
    pip3 install -r /project/requirements.txt && \
    apt-get remove -y \
    gcc \
    libffi-dev \
    libpq-dev \
    libssl-dev && \
    apt-get autoremove -y && \
    apt-get clean
karmux commented 6 years ago

This issue happened with Python 3.6.4 but doesn't happen anymore with Python 3.6.5rc1.

manuel-koch commented 5 years ago

I have the same repeated segfaults in uswgi 2.0.17.1 and python 3.7.1

*** Starting uWSGI 2.0.17.1 (64bit) on [Tue Nov 27 09:01:36 2018] ***
compiled with version: 6.3.0 20170516 on 13 November 2018 15:23:09
os: Linux-4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018
nodename: c1d86efb08d6
machine: x86_64
clock source: unix
pcre jit enabled
detected number of CPU cores: 3
current working directory: /opt/app-root/app
detected binary path: /usr/local/bin/uwsgi
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:8080 fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.7.1 (default, Oct 24 2018, 22:38:59)  [GCC 6.3.0 20170516]
Python main interpreter initialized at 0x5605f0833000
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 5 seconds
mapped 511248 bytes (499 KB) for 5 cores
*** Operational MODE: preforking ***
Installing health check logging filter...
WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x5605f0833000 pid: 1 (default app)
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 19, cores: 1)
spawned uWSGI worker 2 (pid: 20, cores: 1)
running "unix_signal:2 gracefully_kill_them_all" (master-start)...
running "unix_signal:3 gracefully_kill_them_all" (master-start)...
running "unix_signal:5 gracefully_kill_them_all" (master-start)...
running "unix_signal:15 gracefully_kill_them_all" (master-start)...
Python auto-reloader enabled
!!! uWSGI process 19 got Segmentation Fault !!!
*** backtrace of 19 ***
uwsgi(uwsgi_backtrace+0x35) [0x5605f026aca5]
uwsgi(uwsgi_segfault+0x23) [0x5605f026b053]
/lib/x86_64-linux-gnu/libc.so.6(+0x33060) [0x7f85454b2060]
/usr/local/lib/libpython3.7m.so.1.0(PyBytes_AsString+0) [0x7f8545ae9bb0]
uwsgi(uwsgi_python_autoreloader_thread+0x166) [0x5605f0282416]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7494) [0x7f854780f494]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f8545567acf]
*** end of backtrace ***
!!! uWSGI process 20 got Segmentation Fault !!!
*** backtrace of 20 ***
uwsgi(uwsgi_backtrace+0x35) [0x5605f026aca5]
uwsgi(uwsgi_segfault+0x23) [0x5605f026b053]
/lib/x86_64-linux-gnu/libc.so.6(+0x33060) [0x7f85454b2060]
/usr/local/lib/libpython3.7m.so.1.0(PyBytes_AsString+0) [0x7f8545ae9bb0]
uwsgi(uwsgi_python_autoreloader_thread+0x166) [0x5605f0282416]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7494) [0x7f854780f494]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f8545567acf]
*** end of backtrace ***
DAMN ! worker 1 (pid: 19) died, killed by signal 11 :( trying respawn ...
Respawned uWSGI worker 1 (new pid: 23)
DAMN ! worker 2 (pid: 20) died, killed by signal 11 :( trying respawn ...
Python auto-reloader enabled
Respawned uWSGI worker 2 (new pid: 25)