search

openswoole / ext-openswoole

Programmatic server for PHP with async IO, coroutines and fibers
https://openswoole.com
Apache License 2.0
808 stars 51 forks source link

Bug When Calling a Coroutine from Within Loops? #80

Closed benwills closed 2 years ago

benwills commented 2 years ago

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a simple script for reproducing the error.

Using the code below, run it as-is. Then uncomment the three lines in two separate places below, to add a while loop. You'll notice that it does not return the IP addresses.

I am not sure why this would be happening, but I just spent a few hours trying to narrow down why another piece of code wasn't working, and was able to narrow down an example to this.

<?php
Co::set(['dns_server' => '8.8.8.8:53']);

////////////////////////////////////////////////////////////////////////////////
class ObjectGetIp
{
    //----------------------------------------------------------------------------
    public function coroDoTask($cb, $host) {
        Swoole\Coroutine::create($cb, ['scheme' => 'https',
                                       'host'   => $host,
                                       'path'   => '/']);
    }

    //----------------------------------------------------------------------------
    public function download($params) {
        $host = $params['host'];
        echo "LINE: ".__LINE__."\n";
        $ip = Swoole\Coroutine\System::dnsLookup($host, $timeout = 2);
        echo "Found IP: $ip\n\n";
    }

    //----------------------------------------------------------------------------
    public function getHost()
    {
        $hosts = [
            'yahoo.com','msn.com','google.com','bing.com','twitch.tv','qq.com','ebay.com','duckduckgo.com','tiktok.com','paypal.com','aliexpress.com',
        ];
        return [$hosts[rand(0,10)]];
    }

    //----------------------------------------------------------------------------
    public function GetIps()
    {

        // BEGIN: TEST UN/COMMENT OUT THIS LINE
        // while (1) {
        // END: TEST UN/COMMENT OUT THIS LINE

            $x = 5;
            while ($x--) {
                $hosts = $this->getHost();
                foreach ($hosts as $host) {
                    $this->coroDoTask([$this,'download'], $host);
                }
            }

        // BEGIN: TEST UN/COMMENT OUT THESE TWO LINES
        //  sleep(1);
        // }
        // END: TEST UN/COMMENT OUT THESE TWO LINES

    }
}

//==============================================================================
$obj = new ObjectGetIp;
$obj->GetIps();
  1. What did you expect to see?

I expected to see IP addresses.

  1. What did you see instead?

I saw nothing.

  1. What version of Open Swoole are you using (show your php --ri openswoole)? swoole

    Swoole => enabled Author => Swoole Team team@swoole.com Version => 4.8.1 Built => Oct 30 2021 14:45:42 coroutine => enabled with boost asm context epoll => enabled eventfd => enabled signalfd => enabled cpu_affinity => enabled spinlock => enabled rwlock => enabled sockets => enabled openssl => OpenSSL 1.1.1j 16 Feb 2021 dtls => enabled http2 => enabled json => enabled curl-native => enabled pcre => enabled zlib => 1.2.11 brotli => E16777225/D16777225 mutex_timedlock => enabled pthread_barrier => enabled futex => enabled mysqlnd => enabled async_redis => enabled

    Directive => Local Value => Master Value swoole.enable_coroutine => On => On swoole.enable_library => On => On swoole.enable_preemptive_scheduler => Off => Off swoole.display_errors => On => On swoole.use_shortname => On => On swoole.unixsock_buffer_size => 8388608 => 8388608

  2. What is your machine environment used (show your uname -a & php -v & gcc -v) ?

    Linux hostname 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

    PHP 8.0.13 (cli) (built: Nov 22 2021 09:50:43) ( NTS ) Copyright (c) The PHP Group Zend Engine v4.0.13, Copyright (c) Zend Technologies with Zend OPcache v8.0.13, Copyright (c), by Zend Technologies

    Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none:hsa OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

doubaokun commented 2 years ago

Please install according to https://www.swoole.co.uk/docs/get-started/installation and try it again.

Next you have to limit number the coroutines running with https://www.swoole.co.uk/docs/modules/swoole-coroutine-waitgroup

benwills commented 2 years ago

Sounds good. I'll wait for PHP 8.1 to be release in the next day or so, then will go through the install proces you linked to. I'll be back to update after that.

doubaokun commented 2 years ago

Reopen this if you still have issues.

benwills commented 2 years ago

Using php8.1 and installing from pecl, this is still occurring: $ uname -a

Linux hostname 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ php -v

PHP 8.1.0 (cli) (built: Nov 25 2021 20:22:03) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.0, Copyright (c) Zend Technologies
    with Zend OPcache v8.1.0, Copyright (c), by Zend Technologies

$ gcc -v

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

$ php --ri openswoole

openswoole

Open Swoole => enabled
Author => Open Swoole Group & Contributors <hello@swoole.co.uk>
Version => 4.7.2
Built => Nov 25 2021 19:33:17
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
openssl => OpenSSL 1.1.1j  16 Feb 2021
dtls => enabled
pcre => enabled
zlib => 1.2.11
brotli => E16777225/D16777225
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608
benwills commented 2 years ago

And I don't see an option to re-open the issue; presumably becaue I don't have permissions to. I'm assuming you meant to say that I should reply. Let me know if I should create a new issue.

doubaokun commented 2 years ago

OpenSwoole coroutines have to be used within context, according to https://www.swoole.co.uk/docs/modules/swoole-coroutine-run

Working example:

<?php
Co::set(['dns_server' => '8.8.8.8:53']);

////////////////////////////////////////////////////////////////////////////////
class ObjectGetIp
{
    //----------------------------------------------------------------------------
    public function coroDoTask($cb, $host) {
        Swoole\Coroutine::create($cb, ['scheme' => 'https',
                                       'host'   => $host,
                                       'path'   => '/']);
    }

    //----------------------------------------------------------------------------
    public function download($params) {
        $host = $params['host'];
        echo "LINE: ".__LINE__."\n";
        $ip = Swoole\Coroutine\System::dnsLookup($host, $timeout = 2);
        echo "Found IP: $ip\n\n";
    }

    //----------------------------------------------------------------------------
    public function getHost()
    {
        $hosts = [
            'yahoo.com','msn.com','google.com','bing.com','twitch.tv','qq.com','ebay.com','duckduckgo.com','tiktok.com','paypal.com','aliexpress.com',
        ];
        return [$hosts[rand(0,10)]];
    }

    //----------------------------------------------------------------------------
    public function GetIps()
    {

        // BEGIN: TEST UN/COMMENT OUT THIS LINE
        while (1) {
        // END: TEST UN/COMMENT OUT THIS LINE

            $x = 5;
            while ($x--) {
                $hosts = $this->getHost();
                foreach ($hosts as $host) {
                    $this->coroDoTask([$this,'download'], $host);
                }
            }

        // BEGIN: TEST UN/COMMENT OUT THESE TWO LINES
            sleep(1);
        }
        // END: TEST UN/COMMENT OUT THESE TWO LINES

    }
}

//==============================================================================

Co\run(function() {
    $obj = new ObjectGetIp;
    $obj->GetIps();
});
benwills commented 2 years ago

Thank you.