swoole / swoole-src

🚀 Coroutine-based concurrency library for PHP
https://www.swoole.com
Apache License 2.0
18.4k stars 3.16k forks source link

segmentation fault for specific response sizes under HTTP2, coroutines disabled #4365

Closed gwilym closed 3 years ago

gwilym commented 3 years ago
  1. What did you do? If possible, provide a simple script for reproducing the error.

Edit: enable_reuse_port=true might not be necessary, this might just be for any response > 65536 bytes.

<?php

$listenHost = "127.0.0.1";
$listenPort = 9501;

$http = new Swoole\Http\Server($listenHost, $listenPort);
$http->set([
    'open_http2_protocol' => true,
    'enable_reuse_port' => true,
    'enable_coroutine' => false,
    'log_level' => 1,
    'log_file' => 'swoole.log',
]);

error_log("listening on $listenHost:$listenPort");

$http->on('request', function ($request, $response) {
    $response->end(str_repeat('x', 65537));
});
$http->start();
curl -s --http2-prior-knowledge http://127.0.0.1:9501
  1. What did you expect to see?

A response with x repeated 65,537 times.

  1. What did you see instead?

No response, worker crashes with segfault.

[2021-08-10 09:27:23 $55870.0]  WARNING check_worker_exit_status: worker#4[pid=55875] abnormal exit, status=0, signal=11

Note: New Relic is present in this (production) core dump trace below, but, it happens without New Relic. See repro script.

#0  0x00007fb3b896ea7b in swoole::Coroutine::yield (this=0x0) at /tmp/tmp-build/swoole-4.6.7/src/coroutine/base.cc:57
#1  0x00007fb3b89037b3 in swoole_http2_server_respond (ctx=ctx@entry=0x55bc6055e8e0, body=body@entry=0x7fff9cacb330)
    at /tmp/tmp-build/swoole-4.6.7/ext-src/swoole_http2_server.cc:542
#2  0x00007fb3b89040a7 in swoole::http::Context::http2_end (this=0x55bc6055e8e0, zdata=<optimized out>, return_value=0x7fff9cacb388)
    at /tmp/tmp-build/swoole-4.6.7/ext-src/swoole_http2_server.cc:1048
#3  0x000055bc5d974d93 in ?? ()
#4  0x000055bc5db9d27d in execute_ex ()
#5  0x00007fb3b90b2560 in nr_php_execute () at /home/hudson/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1329
#6  0x000055bc5d974d4b in ?? ()
#7  0x000055bc5db9d27d in execute_ex ()
#8  0x00007fb3b90b2560 in nr_php_execute () at /home/hudson/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1329
#9  0x000055bc5d974d4b in ?? ()
#10 0x000055bc5db9d27d in execute_ex ()
#11 0x00007fb3b90b2560 in nr_php_execute () at /home/hudson/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1329
#12 0x000055bc5d974d4b in ?? ()
#13 0x000055bc5db9d27d in execute_ex ()
#14 0x00007fb3b90b2560 in nr_php_execute () at /home/hudson/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1329
#15 0x000055bc5db0ed06 in zend_call_function ()
#16 0x00007fb3b88e27f8 in sw_zend_call_function_ex (retval=0x0, params=0x7fff9cacb820, param_count=2, fci_cache=<optimized out>, function_name=0x0)
    at /tmp/tmp-build/swoole-4.6.7/ext-src/php_swoole_private.h:940
#17 zend::function::call (fci_cache=<optimized out>, argc=argc@entry=2, argv=argv@entry=0x7fff9cacb890, retval=retval@entry=0x0, enable_coroutine=<optimized out>)
    at /tmp/tmp-build/swoole-4.6.7/ext-src/php_swoole_cxx.cc:74
#18 0x00007fb3b8903f4b in swoole_http2_onRequest (client=<optimized out>, stream=0x55bc6036c7e0) at /tmp/tmp-build/swoole-4.6.7/include/swoole_server.h:996
#19 0x00007fb3b89043c1 in swoole_http2_server_parse (client=0x55bc60521210, buf=<optimized out>) at /tmp/tmp-build/swoole-4.6.7/ext-src/swoole_http2_server.cc:933
#20 0x00007fb3b8905482 in swoole_http2_server_onFrame (serv=serv@entry=0x55bc5fd6e890, conn=conn@entry=0x7fb3961ffa98, req=req@entry=0x7fff9cacbb00)
    at /tmp/tmp-build/swoole-4.6.7/ext-src/swoole_http2_server.cc:1024
#21 0x00007fb3b89150ce in php_swoole_http_server_onReceive (serv=0x55bc5fd6e890, req=0x7fff9cacbb00) at /tmp/tmp-build/swoole-4.6.7/ext-src/swoole_http_server.cc:61
#22 0x00007fb3b89bc045 in std::function<int (swoole::Server*, swoole::RecvData*)>::operator()(swoole::Server*, swoole::RecvData*) const (__args#1=<optimized out>,
    __args#0=<optimized out>, this=0x55bc5fd6ee30) at /usr/include/c++/8/bits/std_function.h:682
#23 swoole::Worker_do_task (callback=..., task=0x7fb3a84de010, worker=0x7fb3b955d090, serv=0x55bc5fd6e890) at /tmp/tmp-build/swoole-4.6.7/src/server/worker.cc:213
#24 swoole::Server::accept_task (this=0x55bc5fd6e890, task=0x7fb3a84de010) at /tmp/tmp-build/swoole-4.6.7/src/server/worker.cc:235
#25 0x00007fb3b89bd085 in swoole::Worker_onPipeReceive (reactor=<optimized out>, event=0x7fff9cacbc50) at /tmp/tmp-build/swoole-4.6.7/src/server/worker.cc:702
#26 0x00007fb3b89a30a1 in swoole::ReactorEpoll::wait (this=0x55bc600a6c20, timeo=<optimized out>) at /tmp/tmp-build/swoole-4.6.7/src/reactor/epoll.cc:218
#27 0x00007fb3b89bc9b2 in swoole::Reactor::wait (this=0x55bc600a7060, timeout=0x0) at /tmp/tmp-build/swoole-4.6.7/include/swoole_reactor.h:159
#28 swoole::Server::start_event_worker (this=this@entry=0x55bc5fd6e890, worker=worker@entry=0x7fb3b955d090) at /tmp/tmp-build/swoole-4.6.7/src/server/worker.cc:576
#29 0x00007fb3b89a654f in swoole::Server::spawn_event_worker (this=0x55bc5fd6e890, worker=0x7fb3b955d090) at /tmp/tmp-build/swoole-4.6.7/src/server/manager.cc:622
#30 0x00007fb3b89a7bb1 in swoole::Server::start_manager_process (this=0x55bc5fd6e890) at /tmp/tmp-build/swoole-4.6.7/src/server/manager.cc:164
#31 0x00007fb3b89b2f13 in swoole::ProcessFactory::start (this=0x55bc60050a60) at /tmp/tmp-build/swoole-4.6.7/src/server/process.cc:123
#32 0x00007fb3b89ad2e2 in swoole::Server::start (this=this@entry=0x55bc5fd6e890) at /tmp/tmp-build/swoole-4.6.7/src/server/master.cc:574
#33 0x00007fb3b894ffcd in zim_swoole_server_start (execute_data=<optimized out>, return_value=0x7fff9cacbe68)
    at /tmp/tmp-build/swoole-4.6.7/ext-src/swoole_server.cc:2744
#34 0x000055bc5d974d93 in ?? ()
#35 0x000055bc5db9d27d in execute_ex ()
#36 0x00007fb3b90b2560 in nr_php_execute () at /home/hudson/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1329
#37 0x000055bc5d974d4b in ?? ()
#38 0x000055bc5db9d27d in execute_ex ()
#39 0x00007fb3b90b2560 in nr_php_execute () at /home/hudson/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1329
#40 0x000055bc5d97507f in ?? ()
#41 0x000055bc5db9d27d in execute_ex ()
#42 0x00007fb3b90b2560 in nr_php_execute () at /home/hudson/workspace/php-release-agent/label/centos5-64-nrcamp/agent/php_execute.c:1329
#43 0x000055bc5d97507f in ?? ()
  1. What version of Swoole are you using (show your php --ri swoole)?

Note that I confirmed this also happens with a master build, and php --ri swoole doesn't account for all the server settings (see repro script where coroutines are off).

swoole

Swoole => enabled
Author => Swoole Team <team@swoole.com>
Version => 4.6.7
Built => Aug 10 2021 08:09:18
coroutine => enabled with boost asm context
debug => enabled
trace_log => enabled
kqueue => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.1k  25 Mar 2021
dtls => enabled
http2 => enabled
json => enabled
curl-native => enabled
pcre => enabled
zlib => 1.2.11
brotli => E16777225/D16777225
mysqlnd => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 262144 => 262144
  1. What is your machine environment used (show your uname -a & php -v & gcc -v) ?

Note this also happens in our production Debian environment.

Darwin C02CN25XMD6P 19.6.0 Darwin Kernel Version 19.6.0: Sun Jul  5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64 x86_64

PHP 7.4.22 (cli) (built: Jul 29 2021 18:27:37) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.22, Copyright (c), by Zend Technologies

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.0 (clang-1200.0.26.2)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
gwilym commented 3 years ago

Thanks for the fast fix.

Based on d039fd4, it looks like explicitly setting send_yield => false could be a workaround for users of existing versions.