swoole / swoole-src

🚀 Coroutine-based concurrency library for PHP
https://www.swoole.com
Apache License 2.0
18.48k stars 3.16k forks source link

A bug occurred in Swoole-v5.1.2, please report it #5324

Closed error1001es closed 3 months ago

error1001es commented 6 months ago

Please answer these questions before submitting your issue.

  1. What did you do? If possible, provide a simple script for reproducing the error.

update swoole in docker php ext

  1. What did you expect to see?

all working

  1. What did you see instead?

doesnt response laravel octane with swoole

  1. What version of Swoole are you using (show your php --ri swoole)?

it says 5.1.2

  1. What is your machine environment used (show your uname -a & php -v & gcc -v) ? Linux 1baab60a7013 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 Linux PHP 8.3.7 (cli) (built: May 10 2024 21:59:17) (NTS) Copyright (c) The PHP Group Zend Engine v4.3.7, Copyright (c) Zend Technologies /app $ gcc -v /bin/sh: gcc: not found

[2024-05-12 09:13:10 #19.0] WARNING Server::check_worker_exit_status(): worker(pid=30, id=2) abnormal exit, status=0, signal=11 A bug occurred in Swoole-v5.1.2, please report it. The Swoole developers probably don't know about it, and unless you report it, chances are it won't be fixed. You can read How to report a bug doc before submitting any bug reports:

https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md Please do not send bug reports in the mailing list or personal letters. The issue page is also suitable to submit feature requests.

OS: Linux 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 GCC_VERSION: 13.2.1 20231014 OPENSSL_VERSION: OpenSSL 3.1.4 24 Oct 2023 PHP_VERSION : 8.3.7

error1001es commented 6 months ago

RUN install-php-extensions pcntl sockets swoole pdo_mysql

this command i installed swoole

NathanFreeman commented 6 months ago

Do you have the xdebug extension? If so, try not using it and see.

error1001es commented 6 months ago

no, i dont use xdebug

error1001es commented 6 months ago

it's error only in 5.1.2, 5.1.1 works fine

modrink commented 6 months ago

Same here, using Laravel, Octane, Swoole

Image built using the following Dockerfile:

FROM php:8.3-cli-alpine

WORKDIR /var/www/html

RUN curl -sSLf \
    -o /usr/local/bin/install-php-extensions \
    https://github.com/mlocati/docker-php-extension-installer/releases/download/2.2.11/install-php-extensions \
    && echo "4d8a1bbd2d53a4ce8d194647da9dc2ff37768a26cdb46f459bfc7642d170eb85147534e850bcb503b7b38f04ddb6228f2ef0e50619349ba02fe79aa138452cbe  /usr/local/bin/install-php-extensions" | sha512sum -c - \
    && chmod +x /usr/local/bin/install-php-extensions \
    && install-php-extensions \
    zip \
    pdo_mysql \
    pgsql \
    pdo_pgsql \
    redis \
    calendar \
    gd \
    exif \
    swoole \
    pcntl \
    && apk add --no-cache --virtual .build-deps npm \
    && npm install -g svgo \
    && apk add --no-cache \
    su-exec \
    supervisor \
    jpegoptim \ 
    optipng \
    pngquant \
    gifsicle \
    ffmpeg \
    && apk del .build-deps

Trying to access the project with Octane & Swoole:

[2024-05-13 14:10:43 #31274.0]  WARNING Server::check_worker_exit_status(): worker(pid=31408, id=16) abnormal exit, status=0, signal=11
A bug occurred in Swoole-v5.1.2, please report it.
The Swoole developers probably don't know about it,
and unless you report it, chances are it won't be fixed.
You can read How to report a bug doc before submitting any bug reports:
>> https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md 
Please do not send bug reports in the mailing list or personal letters.
The issue page is also suitable to submit feature requests.

OS: Linux 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1 (2024-04-11) x86_64
GCC_VERSION: 13.2.1 20231014
OPENSSL_VERSION: OpenSSL 3.1.4 24 Oct 2023
PHP_VERSION : 8.3.7

Additional info:

/var/www/html # php --ri swoole

swoole

Swoole => enabled
Author => Swoole Team <team@swoole.com>
Version => 5.1.2
Built => May 13 2024 13:08:15
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
openssl => OpenSSL 3.1.4 24 Oct 2023
dtls => enabled
http2 => enabled
json => enabled
c-ares => 1.27.0
zlib => 1.3.1
brotli => E16781312/D16781312
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_fiber_mock => Off => Off
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608
/var/www/html # uname -a
Linux 6883120929f9 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1 (2024-04-11) x86_64 Linux
/var/www/html # php -v
PHP 8.3.7 (cli) (built: May 10 2024 21:59:17) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.3.7, Copyright (c) Zend Technologies
/var/www/html # gcc -v
/bin/ash: gcc: not found

Can confirm it's working on v5.1.1 (previous build)

NathanFreeman commented 6 months ago

@modrink I can not reproduce it, Is this a consistently reproducible scenario? I am directly accessing 127.0.0.1:8000 without any business logic.

modrink commented 6 months ago

We have Cloudflare & Traefik V3 in front, but I'll test more when I get the chance. I was not able to reproduce this locally (only on server with the proxy). But Swoole v5.1.1 works and OpenSwoole latest works as well.

With Traefik the occurring errors are HTTP 499 and 502

robinnydahl commented 6 months ago

Hi all,

I'm afraid we have not yet had the time to fully make a reproducible case without sharing sensitive details, but I can confirm the issue is present in Swoole 5.1.2 and that reverting back to 5.1.1 solves the issue.

We have tested setting up a clean Laravel project with Octane using Swoole 5.1.2 inside a docker image running from php:8.2-cli-alpine and using mlocati/php-extension-installer to install Swoole. When reaching Swoole on port 8000 the service keeps running, the same goes for when we access it through NGINX inside the image as well.

The NGINX config inside the image looks like this:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 80;
    listen [::]:80;
    server_name domain.com;
    server_tokens off;
    root /home/forge/domain.com/public;

    index index.php;

    charset utf-8;

    location /index.php {
        try_files /not_exists @octane;
    }

    location / {
        try_files $uri $uri/ @octane;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    access_log off;
    error_log  /var/log/nginx/domain.com-error.log error;

    error_page 404 /index.php;

    location @octane {
        set $suffix "";

        if ($uri = /index.php) {
            set $suffix ?$query_string;
        }

        proxy_http_version 1.1;
        proxy_set_header Host $http_host;
        proxy_set_header Scheme $scheme;
        proxy_set_header SERVER_PORT $server_port;
        proxy_set_header REMOTE_ADDR $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        proxy_pass http://127.0.0.1:8000$suffix;
    }
}

The problem actually occurs when we are running the requests through the Kubernetes Ingress Controller which is also using NGINX behind the scenes. After the first request is made through the Kubernetes Ingress Controller the Swoole service shuts down, after this we are also unable to reach the Swoole endpoint directly within the environment as well.

I can concur with @modrink that the issue is that the client closes the connection (HTTP 499), so there seems to be something wrong with the implementation in the HTTP server that was introduced in 5.1.2 and that it's triggered mostly when using HTTP Proxies like NGNIX (with certain settings), Traefik and Cloudflare.

As mentioned we have not yet been able to reproduce this in a way to share yet.

NathanFreeman commented 6 months ago

@robinnydahl @modrink Perhaps some commits led to this issue.I'll take some time to test it.

linuxd3v commented 6 months ago

This happens to me as well. swoole 5.1.2 + xdebug 3.3.2

sandbox-sb.34i5g-swoole8.1 | [2024-05-17 13:07:00 #1.0] WARNING Server::check_worker_exit_status(): worker(pid=154, id=9) abnormal exit, status=0, signal=11 sandbox-sb.34i5g-swoole8.1 | A bug occurred in Swoole-v5.1.2, please report it. sandbox-sb.34i5g-swoole8.1 | The Swoole developers probably don't know about it, sandbox-sb.34i5g-swoole8.1 | and unless you report it, chances are it won't be fixed. sandbox-sb.34i5g-swoole8.1 | You can read How to report a bug doc before submitting any bug reports: sandbox-sb.34i5g-swoole8.1 | >> https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md sandbox-sb.34i5g-swoole8.1 | Please do not send bug reports in the mailing list or personal letters. sandbox-sb.34i5g-swoole8.1 | The issue page is also suitable to submit feature requests. sandbox-sb.34i5g-swoole8.1 | sandbox-sb.34i5g-swoole8.1 | OS: Linux 6.8.8-300.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Apr 27 17:53:31 UTC 2024 x86_64 sandbox-sb.34i5g-swoole8.1 | GCC_VERSION: 11.4.0 sandbox-sb.34i5g-swoole8.1 | OPENSSL_VERSION: OpenSSL 3.0.2 15 Mar 2022 sandbox-sb.34i5g-swoole8.1 | PHP_VERSION : 8.1.2-1ubuntu2.17 sandbox-sb.34i5g-swoole8.1 |

NathanFreeman commented 6 months ago

@linuxd3v Can you provide a simple script for reproducing the error?

NathanFreeman commented 6 months ago

@linuxd3v I found that the max_nesting_level parameter of xdebug is set to 512. If a certain loop exceeds 512 times, a PHP Fatal error will be thrown, terminating the process.

linuxd3v commented 6 months ago

@NathanFreeman I've tried setting max_nesting_level to 50000 - but still seeing the same issue.
Actually - seeing the same issue with Swoole-v5.1.0 and Swoole-v5.1.1 Also tried xdebug 3.3.1 - but with the same result as xdebug 3.3.2.

NathanFreeman commented 6 months ago

@linuxd3v I don't have a good solution for xdebug. There are too many configurations for xdebug, unless you can find out through the logs which specific configuration is causing PHP to throw a fatal error.

NathanFreeman commented 6 months ago

@error1001es @robinnydahl @modrink I can not reproduce it. Please use valgrind or gdb to trace the program execution. https://wiki.swoole.com/en/#/other/issue?id=about-segmentation-fault-core-dump https://wiki.swoole.com/en/#/other/issue?id=memory-analysis-highly-recommended

lv2u commented 6 months ago

@robinnydahl described the exact problem that we are facing with Swoole v5.1.2. I took some time to get the data you requested.

Call stack:

#0  0x0000565154971d20 in zend_hash_real_init_mixed ()
#1  0x0000565154973a35 in zend_hash_str_update ()
#2  0x00007f569d472163 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#3  0x00007f569d52fb70 in swoole_http_parser_execute () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#4  0x00007f569d3dc4ec in php_swoole_http_server_onReceive(swoole::Server*, swoole::RecvData*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#5  0x00007f569d4aecd7 in swoole::Server::worker_accept_event(swoole::DataHead*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#6  0x00007f569d4af8a5 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#7  0x00007f569d4eda91 in swoole::ReactorEpoll::wait(timeval*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#8  0x00007f569d4afe81 in swoole::Server::start_event_worker(swoole::Worker*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#9  0x00007f569d4b3e9b in swoole::Server::spawn_event_worker(swoole::Worker*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#10 0x00007f569d4b6082 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#11 0x00007f569d50ffef in swoole_fork_exec(std::function<void ()> const&) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#12 0x00007f569d4b661c in swoole::Server::start_manager_process() () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#13 0x00007f569d4cd3f9 in swoole::ProcessFactory::start() () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#14 0x00007f569d4c0c8a in swoole::Server::start() () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#15 0x00007f569d47f7e6 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#16 0x00005651549d576d in execute_ex ()
#17 0x00005651549d7883 in zend_execute ()
#18 0x00005651549645a8 in zend_execute_scripts ()
#19 0x00005651548f9e80 in php_execute_script ()
#20 0x0000565154a4d245 in ?? ()
#21 0x0000565154647604 in ?? ()
#22 0x00007f569ebbd70a in libc_start_main_stage2 (main=0x565154647310, argc=3, argv=0x7fffd594ebc8) at src/env/__libc_start_main.c:95
#23 0x000056515464769f in _start ()

It's worth noting that the trace doesn't always include frame 0. Sometimes it stops at zend_hash_str_update () (What is frame 1 in the trace above).

It's also worth noting that instead of PHP 8.2, we're using the php:8.3.7-cli-alpine20 base image. We also run Swoole using Laravel Octane.

Our Dockerfile is based on this example Dockerfile: https://github.com/swoole/docker-swoole/blob/master/dockerfiles/5.1.2/php8.3/alpine/Dockerfile

HallexCosta commented 6 months ago

Please answer these questions before submitting your issue.

  1. What did you do? If possible, provide a simple script for reproducing the error.

update swoole in docker php ext

  1. What did you expect to see?

all working

  1. What did you see instead?

doesnt response laravel octane with swoole

  1. What version of Swoole are you using (show your php --ri swoole)?

it says 5.1.2

  1. What is your machine environment used (show your uname -a & php -v & gcc -v) ? Linux 1baab60a7013 5.15.146.1-microsoft-standard-WSL2 编译出错 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 Linux PHP 8.3.7 (cli) (built: May 10 2024 21:59:17) (NTS) Copyright (c) The PHP Group Zend Engine v4.3.7, Copyright (c) Zend Technologies /app $ gcc -v /bin/sh: gcc: not found

[2024-05-12 09:13:10 #19.0] WARNING Server::check_worker_exit_status(): worker(pid=30, id=2) abnormal exit, status=0, signal=11 A bug occurred in Swoole-v5.1.2, please report it. The Swoole developers probably don't know about it, and unless you report it, chances are it won't be fixed. You can read How to report a bug doc before submitting any bug reports:

https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md Please do not send bug reports in the mailing list or personal letters. The issue page is also suitable to submit feature requests.

OS: Linux 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 GCC_VERSION: 13.2.1 20231014 OPENSSL_VERSION: OpenSSL 3.1.4 24 Oct 2023 PHP_VERSION : 8.3.7

Same problem here.

OS: Ubuntu PHP Version: 8.1.28 Swoole Version: 5.1.2 Binary generator CLI: I'm use Static PHP CLI

Context description: I'm using the hyperf application binary to launch my server application, but the worker die when receiving a request from the browser. Apparently when I use an HTTP client it works normally

Command start: bin/app start image

After make a request in browser (The request seems to go into a loop and the logs show me this error message) image

The same request being done in HTTP Client (or curl) work fine image

NathanFreeman commented 5 months ago

@HallexCosta @lv2u @error1001es @robinnydahl @modrink We have released Swoole v5.1.3, optimized a dynamic array. You can try Swoole v5.1.3 to see if this issue still exists.

salkhwlani commented 5 months ago

@NathanFreeman confirmed the issue has been resolved with v5.1.3 btw the Swoole was serving the request fine directly but failed behind an LB or Nginx

NathanFreeman commented 5 months ago

What error messages did you get? @salkhwlani

salkhwlani commented 5 months ago

@NathanFreeman worker[0] error: exitCode=0, signal=11

NathanFreeman commented 5 months ago

Are you using xdebug, or can you check the stack output using valgrind? @salkhwlani

salkhwlani commented 5 months ago

No, I don't use Xdebug; the issue is resolved once I update the swoole from 5.1.2 to version v5.1.3

lv2u commented 5 months ago

Confirming Swoole v5.1.3 fixes the issues described in this report.

error1001es commented 5 months ago

confirming swoole-5.1.3 fixes it!

HallexCosta commented 5 months ago

Sorry for taking so long to answer. Confirming swoole-5.1.3 fixes it!