search

swoole / swoole-src

🚀 Coroutine-based concurrency library for PHP
https://www.swoole.com
Apache License 2.0
18.41k stars 3.16k forks source link

php8.3 swoole5.1.3 kernel: php[1048884]: segfault at 48 ip 00007f82db25f197 sp 00007ffc466aa9c0 error 4 in swoole.so[7f82db1df000+1a2000] #5410

Closed mdys closed 1 month ago

mdys commented 1 month ago

Please answer these questions before submitting your issue.

  1. What did you do? If possible, provide a simple script for reproducing the error. 使用swoole coroutine+task完成日均百万级数量耗时任务处理，在任务开启后头一小时运行还算正常，之后会越来越慢，然后崩溃卡住。 图片 图片

然后收到如下错误： kernel: php[1048884]: segfault at 48 ip 00007f82db25f197 sp 00007ffc466aa9c0 error 4 in swoole.so[7f82db1df000+1a2000] ` [2024-07-15 09:02:22 #3280935.0] WARNING Server::check_worker_exit_status(): worker(pid=1105515, id=303) abnormal exit, status=0, signal=11 A bug occurred in Swoole-v5.1.3, please report it. The Swoole developers probably don't know about it, and unless you report it, chances are it won't be fixed. You can read How to report a bug doc before submitting any bug reports:

https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md Please do not send bug reports in the mailing list or personal letters. The issue page is also suitable to submit feature requests.

OS: Linux 3.10.0-1160.119.1.el7.x86_64 #1 SMP Tue Jun 4 14:43:51 UTC 2024 x86_64 GCC_VERSION: 10.2.1 20210130 (Red Hat 10.2.1-11) OPENSSL_VERSION: OpenSSL 1.0.2k-fips 26 Jan 2017 PHP_VERSION : 8.3.8 gdb core-php-1062455-1721031782 Type "apropos word" to search for commands related to "word"... BFD: warning: /root/core-php-1062455-1721031782 is truncated: expected core file size >= 3330588672, found: 543755968 BFD: warning: /root/core-php-1062455-1721031782 is truncated: expected core file size >= 3330588672, found: 543755968 [New LWP 1062455] Failed to read a valid object file image from memory. Core was generated by `50001event_swoole4_process_server_worker '. Program terminated with signal SIGSEGV, Segmentation fault.

0 0x00007f82db25f197 in ?? ()

(gdb) f 0

0 0x00007f82db25f197 in ?? ()

(gdb) f 1

0 0x0000000000000000 in ?? ()

(gdb) bt

0 0x00007f82db25f197 in ?? ()

Backtrace stopped: Cannot access memory at address 0x7ffc466aa9c0

`

  1. What did you expect to see? 解决运行异常

  2. What did you see instead?

/var/log/messages 不断增加 Jul 15 09:29:37 localhost kernel: php[1233287]: segfault at 48 ip 00007fb90b3b8197 sp 00007ffdf2188c70 error 4 in swoole.so[7fb90b338000+1a2000] core-dump 文件不断产生错误 core-php-1079730-1721032383

  1. What version of Swoole are you using (show your php --ri swoole)?

Swoole => enabled Author => Swoole Team team@swoole.com Version => 5.1.3 Built => Jun 6 2024 10:37:52 coroutine => enabled with boost asm context trace_log => enabled epoll => enabled eventfd => enabled signalfd => enabled cpu_affinity => enabled spinlock => enabled rwlock => enabled sockets => enabled openssl => OpenSSL 1.0.2k-fips 26 Jan 2017 http2 => enabled json => enabled pcre => enabled c-ares => 1.32.1 zlib => 1.2.7 mutex_timedlock => enabled pthread_barrier => enabled futex => enabled mysqlnd => enabled async_redis => enabled coroutine_odbc => enabled coroutine_sqlite => enabled Directive => Local Value => Master Value swoole.enable_coroutine => On => On swoole.enable_library => On => On swoole.enable_fiber_mock => Off => Off swoole.enable_preemptive_scheduler => Off => Off swoole.display_errors => On => On swoole.use_shortname => On => On swoole.unixsock_buffer_size => 8388608 => 8388608

  1. What is your machine environment used (show your uname -a & php -v & gcc -v) ? CentOS Linux release 7.9.2009 3.10.0-1160.119.1.el7.x86_64 Mem : 256G CPU : 128C PHP 8.3.8 (cli) (built: Jun 4 2024 14:53:17) (NTS gcc x86_64) Copyright (c) The PHP Group Zend Engine v4.3.8, Copyright (c) Zend Technologies with Zend OPcache v8.3.8, Copyright (c), by Zend Technologies [PHP Modules] ast bcmath brotli bz2 calendar Core ctype curl date dba dom enchant exif fileinfo filter ftp gd gettext gmp hash iconv igbinary imagick imap intl json ldap libxml maxminddb mbstring mcrypt msgpack mysqli mysqlnd odbc openssl pcntl pcre PDO pdo_mysql PDO_ODBC pdo_sqlite Phar posix random readline redis Reflection session shmop SimpleXML soap sockets sodium SPL sqlite3 standard swoole sysvmsg sysvsem sysvshm tokenizer xlswriter xml xmlreader xmlrpc xmlwriter xsl Zend OPcache zip zlib [Zend Modules] Zend OPcache

Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-8/root/usr --mandir=/opt/rh/devtoolset-8/root/usr/share/man --infodir=/opt/rh/devtoolset-8/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-8.3.1-20190311/obj-x86_64-redhat-linux/isl-install --disable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux Thread model: posix gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)

NathanFreeman commented 1 month ago

把maxminddb这个扩展注释了试试有没有问题

mdys commented 1 month ago

ok 感谢。 我取消观察下

mdys commented 1 month ago

maxminddb 设置了 'max_request' => 0, 另外请问下 这种情况如何可以避免呢？我这边有websocket服务，不希望中途worker中断，请问这个需要怎么操作啊？在运行半天左右就会出现一次。谢谢！ [2024-07-16 05:32:02 #2018835.0] WARNING Server::check_worker_exit_status(): worker(pid=2888018, id=281) abnormal exit, status=255, signal=0 [2024-07-16 05:32:34 #2018835.0] INFO Server is reloading all workers now [2024-07-16 05:32:35 *2019357.0] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *2019365.4] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *2019361.2] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *2019359.1] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *2019363.3] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *2019367.5] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *3189624.0] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *3189630.7] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *3189631.5] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 *3189625.4] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10

firedtoad commented 1 month ago

[image: image.png] 好富裕的大佬，你core size 开的太小了 堆栈不全

mdys @.***> 于2024年7月16日周二 14:26写道：

maxminddb 设置了 'max_request' => 0, 另外请问下 这种情况如何可以避免呢？我这边有websocket服务，不希望中途worker中断，请问这个需要怎么操作啊？在运行半天左右就会出现一次。谢谢！ [2024-07-16 05:32:02 #2018835.0] WARNING Server::check_worker_exit_status(): worker(pid=2888018, id=281) abnormal exit, status=255, signal=0 [2024-07-16 05:32:34 #2018835.0] INFO Server is reloading all workers now [2024-07-16 05:32:35 2019357.0] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 2019365.4] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 2019361.2] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 2019359.1] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 2019363.3] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 2019367.5] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 3189624.0] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 3189630.7] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 3189631.5] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10 [2024-07-16 05:32:35 3189625.4] WARNING swoole_signalfd_event_callback() (ERRNO 721): Unable to find callback function for signal User defined signal 1: 10

— Reply to this email directly, view it on GitHub https://github.com/swoole/swoole-src/issues/5410#issuecomment-2230117793, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK6QJX33DHW7TIBJG7W2RLZMS4K3AVCNFSM6AAAAABK4IORM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZQGEYTONZZGM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

mdys commented 1 month ago

感谢指点，请问这个怎么优化下呢？产生的core-dump文件有2个多G。

firedtoad commented 1 month ago

ulimit -c unlimited 再复现一下看一下宕在哪里

mdys @.***> 于2024年7月16日周二 17:29写道：

感谢指点，请问这个怎么优化下呢？产生的core-dump文件有2个多G。

— Reply to this email directly, view it on GitHub https://github.com/swoole/swoole-src/issues/5410#issuecomment-2230439303, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK6QJWYLTVXLJMOJAM3SEDZMTRWXAVCNFSM6AAAAABK4IORM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZQGQZTSMZQGM . You are receiving this because you commented.Message ID: @.***>

mdys commented 1 month ago

再帮看下 感谢 图片

firedtoad commented 1 month ago

你这是内存释放的问题

mdys @.***> 于2024年7月16日周二 20:20写道：

再帮看下 感谢 default.png (view on web) https://github.com/user-attachments/assets/201af2e7-59f5-4ffd-ab84-236d4272b65f

— Reply to this email directly, view it on GitHub https://github.com/swoole/swoole-src/issues/5410#issuecomment-2230750428, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK6QJRSG5B6LYCHOZUUJWDZMUFY7AVCNFSM6AAAAABK4IORM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZQG42TANBSHA . You are receiving this because you commented.Message ID: @.***>

firedtoad commented 1 month ago

你安装一下调试符号，看一下行号

哈蟆 @.***> 于2024年7月16日周二 20:30写道：

你这是内存释放的问题

mdys @.***> 于2024年7月16日周二 20:20写道：

再帮看下 感谢 default.png (view on web) https://github.com/user-attachments/assets/201af2e7-59f5-4ffd-ab84-236d4272b65f

— Reply to this email directly, view it on GitHub https://github.com/swoole/swoole-src/issues/5410#issuecomment-2230750428, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK6QJRSG5B6LYCHOZUUJWDZMUFY7AVCNFSM6AAAAABK4IORM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZQG42TANBSHA . You are receiving this because you commented.Message ID: @.***>

firedtoad commented 1 month ago

盲猜一个原因，你编译swoole的机器和执行代码的机器不是同一个操作系统环境 glibc版本不匹配

mdys commented 1 month ago

是的，我的GLIBC 是后来升级到2.31得， swoole是 直接yum安装的。 不好意思我没有gdb调试的经验，请问是这样么？ 你安装一下调试符号，看一下行号 图片

mdys commented 1 month ago

盲猜一个原因，你编译swoole的机器和执行代码的机器不是同一个操作系统环境 我试试重新根据现在的环境编译一份看看 -_-" 感谢指点迷津

firedtoad commented 1 month ago

问题确认，是glibc版本不匹配导致的

mdys @.***> 于2024年7月16日周二 20:48写道：

是的，我的GLIBC 是后来升级到2.31得， swoole是 直接yum安装的。 不好意思我没有gdb调试的经验，请问是这样么？ 你安装一下调试符号，看一下行号 default.png (view on web) https://github.com/user-attachments/assets/6c0a31a7-08f8-42d2-9ad8-ab6339f9612b

— Reply to this email directly, view it on GitHub https://github.com/swoole/swoole-src/issues/5410#issuecomment-2230806759, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK6QJUTDPNOIS6OSZOCTFDZMUJBPAVCNFSM6AAAAABK4IORM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZQHAYDMNZVHE . You are receiving this because you commented.Message ID: @.***>

mdys commented 1 month ago

感谢大佬解惑！！！ 非常感谢，我重新编译下 ，困扰我老久了