swoft-cloud / swoft

🚀 PHP Microservice Full Coroutine Framework
https://swoft.org
Apache License 2.0
5.58k stars 788 forks source link

操作MySQL会引发内存泄漏,用定时器频繁执行每分钟内存占用大约增加1Mb #578

Closed jqhph closed 5 years ago

jqhph commented 5 years ago
Q A
Bug report? yes
Feature request? yes/no
Swoft version 当前最新版本,"swoft/redis:v1.0.15", "swoft/db:v1.1.26"
Swoole version 4.2.12
PHP version 7.2.11
Runtime environment Docker

Details

Describe what you are trying to achieve and what goes wrong.

使用 docker stats监控容器内存情况,开始:

CONTAINER ID        NAME               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
d930f47f6504        swofttest          68.24MiB / 15.56GiB   0.83%               8.73MB / 17.6MB     2.52MB / 287kB      6

1小时后,容器内存占用达到133.6MiB每分钟大约增加1MB内存,非常严重

CONTAINER ID        NAME               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
d930f47f6504        swofttest          133.6MiB / 15.56GiB   0.83%               8.73MB / 17.6MB     2.52MB / 287kB      6

Provide minimal script to reproduce the issue

/**
 * @ServerListener(SwooleEvent::ON_WORKER_START)
 */
class WorkerStartListener implements WorkerStartInterface
{
    /**
     * @param Server $server
     * @param int $workerId
     * @param bool $isWorker
     */
    public function onWorkerStart(Server $server, int $workerId, bool $isWorker)
    {
        if (!$isWorker) {
            return;
        }

        if ($server->worker_id > 0) {
            return;
        }

        $data = range(0, 100);

        swoole_timer_tick(100, function () use ($data) {
            go(function () use ($data) {
                Query::table(Node::class)
                    ->where('id', 7)
                    ->limit(1)
                    ->update(['name' => '192.168.0.2'])
                    ->getResult();

                // bean(Redis::class)->hGet('test', '192.168.0.2');

                RequestContext::destroy();
            });
        });

    }
}
jqhph commented 5 years ago

dockerfile如下:

FROM php:7.2

MAINTAINER huangzhhui <h@swoft.org>

# Version
ENV PHPREDIS_VERSION 4.0.0
ENV HIREDIS_VERSION 0.13.3
ENV SWOOLE_VERSION 4.2.12

# Timezone
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
    && echo 'Asia/Shanghai' > /etc/timezone

# Libs
RUN apt-get update \
    && apt-get install -y \
        curl \
        wget \
        git \
        zip \
        libz-dev \
        libssl-dev \
        libnghttp2-dev \
        libpcre3-dev \
        libjpeg-dev \
        libjpeg62-turbo-dev \
        libfreetype6-dev \
        libpng-dev \
    && apt-get clean \
    && apt-get autoremove

# Composer
RUN curl -sS https://getcomposer.org/installer | php \
    && mv composer.phar /usr/local/bin/composer \
    && composer self-update --clean-backups

# GD扩展
RUN docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/
RUN docker-php-ext-install gd

# PDO extension
RUN docker-php-ext-install pdo_mysql

# Bcmath extension
RUN docker-php-ext-install bcmath

# Zip extension
RUN docker-php-ext-install zip

# SOCKETS扩展
RUN docker-php-ext-install sockets

# Redis extension
RUN wget http://pecl.php.net/get/redis-${PHPREDIS_VERSION}.tgz -O /tmp/redis.tar.tgz \
    && pecl install /tmp/redis.tar.tgz \
    && rm -rf /tmp/redis.tar.tgz \
    && docker-php-ext-enable redis

# Hiredis
RUN wget https://github.com/redis/hiredis/archive/v${HIREDIS_VERSION}.tar.gz -O hiredis.tar.gz \
    && mkdir -p hiredis \
    && tar -xf hiredis.tar.gz -C hiredis --strip-components=1 \
    && rm hiredis.tar.gz \
    && ( \
        cd hiredis \
        && make -j$(nproc) \
        && make install \
        && ldconfig \
    ) \
    && rm -r hiredis

# Swoole extension
RUN wget https://github.com/swoole/swoole-src/archive/v${SWOOLE_VERSION}.tar.gz -O swoole.tar.gz \
    && mkdir -p swoole \
    && tar -xf swoole.tar.gz -C swoole --strip-components=1 \
    && rm swoole.tar.gz \
    && ( \
        cd swoole \
        && phpize \
        && ./configure --enable-async-redis --enable-mysqlnd --enable-openssl --enable-http2 \
        && make -j$(nproc) \
        && make install \
    ) \
    && rm -r swoole \
    && docker-php-ext-enable swoole

# Mongodb extension
RUN wget http://pecl.php.net/get/mongodb-1.5.2.tgz -O mongodb.tgz \
    && mkdir -p mongodb \
    && tar -xf mongodb.tgz -C mongodb --strip-components=1 \
    && rm mongodb.tgz \
    && ( \
        cd mongodb \
        cd mongodb-1.5.2 \
        && phpize \
        && ./configure --with-mongodb-ssl \
        && make -j$(nproc) \
        && make install \
    ) \
    && rm -r mongodb \
    && docker-php-ext-enable mongodb

MAINTAINER harry

ADD . /var/www/swoft

WORKDIR /var/www/swoft

EXPOSE 80

CMD ["php", "/var/www/swoft/bin/swoft", "start"]
jqhph commented 5 years ago

同样环境,单纯只使用swoole的情况下测试同样的逻辑,内存浮动始终在8MiB上下。说明内存泄漏是swoft的问题,测试脚本如下:

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
1a0aef76078b        swooletest          0.22%               8.786MiB / 15.56GiB   0.05%               2.52MB / 4.27MB     0B / 0B             1
<?php

$serv = new Swoole\Server('0.0.0.0', 9501, SWOOLE_BASE, SWOOLE_SOCK_TCP);

$serv->set(array(
    'worker_num' => 1,
    'daemonize' => 0,
    'backlog' => 128,
));

$serv->on('Receive', function () {

});

$serv->on('WorkerStart', function ($serv, $worker_id) {
    var_dump('测试内存泄漏进程启动');

    $data = range(0, 100);

    $db = new Swoole\Coroutine\Mysql();
    $db->connect([
        'host' => '192.168.0.144',
        'port' => 3306,
        'user' => 'root',
        'password' => '123456',
        'database' => 'tbtest',
    ]);

    $redis = new Swoole\Coroutine\Redis();
    $redis->connect('192.168.0.144', 6379, 1);

    $val = $redis->hSet('test', 't', 'HELLO');

    swoole_timer_tick(100, function () use ($data, $db, $redis) {
        go(function () use ($data, $db, $redis) {
            $db->setDefer(1);
            $db->query('UPDATE `node` SET `option`="0" WHERE `id`="25"');
            $db->recv();
            $affected = $db->affected_rows;

            // $val = $redis->hGet('test', 't');
        });
    });
});

$serv->start();
jqhph commented 5 years ago

重新测试了下,发现可能与redis操作无关

huangzhhui commented 5 years ago

定时器内需自行执行 RequestContext::destory() 清理协程上下文

jqhph commented 5 years ago

定时器内需自行执行 RequestContext::destory() 清理协程上下文

执行过了,你看上面给的例子

jqhph commented 5 years ago

@huangzhhui @inhere 我目测找到内存泄漏的主要原因,需要在逻辑最后面追加一句App::getLogger()->appendNoticeLog();,2.0版本能否避开这样的问题?

swoole_timer_tick(100, function () use ($data) {
            go(function () use ($data) {
                Db::query('UPDATE `node` SET `option`=:option WHERE `id`=:id', [
                    ':option' => '0',
                    ':id' => 25,
                ])->getResult();

                RequestContext::destroy();
                App::getLogger()->appendNoticeLog();
            });
        });
xnnyeYp commented 5 years ago

这个问题在worker和task中也会发生,当请求过于多时会发生内存一直增加,直到超出PHP的memory_limit,压测时可以复现这个问题。

stelin commented 5 years ago

@jqhph 2.0已发布,建议升级到2.0