amphp / parallel-functions

Simplified parallel processing for PHP based on Amp.
https://amphp.org/parallel-functions
MIT License
271 stars 18 forks source link

Segmentation fault during 10 parallel processes execution #3

Closed optiman closed 6 years ago

optiman commented 6 years ago

If I spawn 10 (or more) processes, regardless what they do (sleeping for 5 seconds in this case) - I'm receiving segmentation fault at the end, even though spawned processes seem to exit properly. When spawning 8 or less processes, everything is fine.

PHP Version => 7.2.0-2+ubuntu16.04.1+deb.sury.org+2

Code:

$start = microtime(true);
wait(parallelMap(array_fill(0, 10, 5), 'sleep'));
print 'Took ' . (microtime(true) - $start) . ' milliseconds.' . PHP_EOL;

Result:

Took 5.22438788414 milliseconds.
[1]    12088 segmentation fault (core dumped)  php 1-simple-function.php

Backtrace:

#0  0x00005604d1507124 in ZEND_INIT_METHOD_CALL_SPEC_TMPVAR_CONST_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:49824
#1  0x00005604d151046c in execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:63235
#2  0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#3  execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#4  0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#5  execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#6  0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#7  execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#8  0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#9  execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#10 0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#11 execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#12 0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#13 execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#14 0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#15 execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#16 0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#17 execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#18 0x00005604d1515934 in ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:888
#19 execute_ex (ex=0x8) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_vm_execute.h:59752
#20 0x00005604d1456002 in zend_call_function (fci=0x7f7e6b21c980, fci@entry=0x7ffd14e04460, fci_cache=fci_cache@entry=0x7ffd14e04430)
    at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_execute_API.c:817
#21 0x00005604d1485334 in zend_call_method (object=object@entry=0x7ffd14e04540, obj_ce=<optimized out>, fn_proxy=fn_proxy@entry=0x7ffd14e04538, 
    function_name=function_name@entry=0x5604d1572690 "__destruct", function_name_len=function_name_len@entry=10, retval_ptr=retval_ptr@entry=0x0, param_count=0, arg1=0x0, arg2=0x0)
    at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_interfaces.c:100
#22 0x00005604d14a0542 in zend_objects_destroy_object (object=<optimized out>) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_objects.c:146
#23 0x00005604d14a5436 in zend_objects_store_call_destructors (objects=objects@entry=0x5604d1889a98 <executor_globals+824>) at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_objects_API.c:58
#24 0x00005604d145487b in shutdown_destructors () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend_execute_API.c:239
#25 0x00005604d1466307 in zend_call_destructors () at /build/php7.2-UPykvD/php7.2-7.2.0/Zend/zend.c:1021
#26 0x00005604d1400625 in php_request_shutdown (dummy=<optimized out>) at /build/php7.2-UPykvD/php7.2-7.2.0/main/main.c:1853
#27 0x00005604d151948c in do_cli (argc=2, argv=0x5604d2abe290) at /build/php7.2-UPykvD/php7.2-7.2.0/sapi/cli/php_cli.c:1178
#28 0x00005604d12b6e0c in main (argc=2, argv=0x5604d2abe290) at /build/php7.2-UPykvD/php7.2-7.2.0/sapi/cli/php_cli.c:1404
kelunik commented 6 years ago

We can't really do anything about segfaults, please report them to bugs.php.net. Do you have xdebug or pthreads enabled?

optiman commented 6 years ago

No, xdebug and pthreads are not enabled.

trowski commented 6 years ago

Tried this locally on 7.2.0 (with and without xdebug) and had no issue making the second parameter very large. We've been seeing segfaults with any code using amphp/process when usingxdebug or phpdbg on travis. Maybe the stacktrace you provided would be useful to @bwoebi for debugging.

optiman commented 6 years ago

Ok, I think I found the problem - it's ev extension. I disabled it and segfault is gone with both stream_select and uv. Sorry for bothering you all, I'm closing this issue as it is not related to the library itself.

kelunik commented 6 years ago

@optiman If you have time, could you test whether the patch mentioned in https://bitbucket.org/osmanov/pecl-ev/issues/31/segfault-in-ev_timer_stop fixes the segfault with ev enabled? I didn't have time to test it and then forgot about it.

optiman commented 6 years ago

Yes, it fixes segfault but adds a notice for every spawned process:

PHP Notice:  Trying to get property 'is_active' of non-object

Here:

    public function __destruct()
    {
//        foreach ($this->events as $event) {
//            $event->stop();
        foreach ($this->events as &$event) {
            if ($event->is_active) {
                $event->stop();
                $event = null;
            }
        }
    }

$event is null.

kelunik commented 6 years ago

Could you var_dump($this->events);?

optiman commented 6 years ago
var_dump($this->events);
array(3) {
  ["b"]=>
  NULL
  ["c"]=>
  object(EvIo)#45 (6) {
    ["fd"]=>
    resource(73) of type (stream)
    ["events"]=>
    int(1)
    ["is_active"]=>
    bool(true)
    ["data"]=>
    object(Amp\Loop\Watcher)#44 (8) {
      ["type"]=>
      int(1)
      ["enabled"]=>
      bool(true)
      ["referenced"]=>
      bool(false)
      ["id"]=>
      string(1) "c"
      ["callback"]=>
      object(Closure)#43 (2) {
        ["static"]=>
        array(4) {
          ["deferred"]=>
          &object(Amp\Deferred)#31 (4) {
            ["resolved":"Amp\Deferred":private]=>
            bool(false)
            ["result":"Amp\Deferred":private]=>
            NULL
            ["onResolved":"Amp\Deferred":private]=>
            object(Closure)#63 (2) {
              ["this"]=>
              object(Amp\Coroutine)#61 (9) {
                ["generator":"Amp\Coroutine":private]=>
                object(Generator)#60 (0) {
                }
                ["onResolve":"Amp\Coroutine":private]=>
                *RECURSION*
                ["immediate":"Amp\Coroutine":private]=>
                bool(true)
                ["exception":"Amp\Coroutine":private]=>
                NULL
                ["value":"Amp\Coroutine":private]=>
                NULL
                ["resolved":"Amp\Coroutine":private]=>
                bool(false)
                ["result":"Amp\Coroutine":private]=>
                NULL
                ["onResolved":"Amp\Coroutine":private]=>
                object(Closure)#64 (2) {
                  ["this"]=>
                  object(Amp\Coroutine)#54 (9) {
                    ["generator":"Amp\Coroutine":private]=>
                    object(Generator)#53 (0) {
                    }
                    ["onResolve":"Amp\Coroutine":private]=>
                    *RECURSION*
                    ["immediate":"Amp\Coroutine":private]=>
                    bool(true)
                    ["exception":"Amp\Coroutine":private]=>
                    NULL
                    ["value":"Amp\Coroutine":private]=>
                    NULL
                    ["resolved":"Amp\Coroutine":private]=>
                    bool(false)
                    ["result":"Amp\Coroutine":private]=>
                    NULL
                    ["onResolved":"Amp\Coroutine":private]=>
                    object(Closure)#65 (1) {
                      ["parameter"]=>
                      array(1) {
                        ["$exception"]=>
                        string(10) ""
                      }
                    }
                    ["resolutionTrace":"Amp\Coroutine":private]=>
                    NULL
                  }
                  ["parameter"]=>
                  array(2) {
                    ["$exception"]=>
                    string(10) ""
                    ["$value"]=>
                    string(10) ""
                  }
                }
                ["resolutionTrace":"Amp\Coroutine":private]=>
                NULL
              }
              ["parameter"]=>
              array(2) {
                ["$exception"]=>
                string(10) ""
                ["$value"]=>
                string(10) ""
              }
            }
            ["resolutionTrace":"Amp\Deferred":private]=>
            NULL
          }
          ["readable"]=>
          &bool(true)
          ["chunkSize"]=>
          int(8192)
          ["useFread"]=>
          bool(true)
        }
        ["parameter"]=>
        array(2) {
          ["$watcher"]=>
          string(10) ""
          ["$stream"]=>
          string(10) ""
        }
      }
      ["data"]=>
      NULL
      ["value"]=>
      resource(55) of type (stream)
      ["__propertySuggestThreshold":"Amp\Loop\Watcher":private]=>
      int(70)
    }
    ["is_pending"]=>
    bool(false)
    ["priority"]=>
    int(0)
  }
  ["e"]=>
  object(EvIo)#32 (6) {
    ["fd"]=>
    resource(74) of type (stream)
    ["events"]=>
    int(1)
    ["is_active"]=>
    bool(true)
    ["data"]=>
    object(Amp\Loop\Watcher)#46 (8) {
      ["type"]=>
      int(1)
      ["enabled"]=>
      bool(true)
      ["referenced"]=>
      bool(false)
      ["id"]=>
      string(1) "e"
      ["callback"]=>
      array(2) {
        [0]=>
        string(33) "Amp\Process\Internal\Posix\Runner"
        [1]=>
        string(33) "onProcessEndExtraDataPipeReadable"
      }
      ["data"]=>
      object(Amp\Process\Internal\Posix\Handle)#23 (12) {
        ["joinDeferred"]=>
        object(Amp\Deferred)#25 (4) {
          ["resolved":"Amp\Deferred":private]=>
          bool(false)
          ["result":"Amp\Deferred":private]=>
          NULL
          ["onResolved":"Amp\Deferred":private]=>
          NULL
          ["resolutionTrace":"Amp\Deferred":private]=>
          NULL
        }
        ["proc"]=>
        resource(57) of type (process)
        ["extraDataPipe"]=>
        resource(56) of type (stream)
        ["extraDataPipeWatcher"]=>
        string(1) "e"
        ["extraDataPipeStartWatcher"]=>
        string(1) "d"
        ["originalParentPid"]=>
        int(22454)
        ["stdin"]=>
        object(Amp\Process\ProcessOutputStream)#27 (4) {
          ["queuedWrites":"Amp\Process\ProcessOutputStream":private]=>
          array(0) {
          }
          ["shouldClose":"Amp\Process\ProcessOutputStream":private]=>
          bool(false)
          ["resourceStream":"Amp\Process\ProcessOutputStream":private]=>
          object(Amp\ByteStream\ResourceOutputStream)#35 (5) {
            ["resource":"Amp\ByteStream\ResourceOutputStream":private]=>
            &resource(53) of type (stream)
            ["watcher":"Amp\ByteStream\ResourceOutputStream":private]=>
            string(1) "a"
            ["writes":"Amp\ByteStream\ResourceOutputStream":private]=>
            object(SplQueue)#36 (2) {
              ["flags":"SplDoublyLinkedList":private]=>
              int(4)
              ["dllist":"SplDoublyLinkedList":private]=>
              array(0) {
              }
            }
            ["writable":"Amp\ByteStream\ResourceOutputStream":private]=>
            &bool(true)
            ["chunkSize":"Amp\ByteStream\ResourceOutputStream":private]=>
            NULL
          }
          ["error":"Amp\Process\ProcessOutputStream":private]=>
          NULL
        }
        ["stdout"]=>
        object(Amp\Process\ProcessInputStream)#30 (5) {
          ["initialRead":"Amp\Process\ProcessInputStream":private]=>
          NULL
          ["shouldClose":"Amp\Process\ProcessInputStream":private]=>
          bool(false)
          ["referenced":"Amp\Process\ProcessInputStream":private]=>
          bool(true)
          ["resourceStream":"Amp\Process\ProcessInputStream":private]=>
          object(Amp\ByteStream\ResourceInputStream)#39 (4) {
            ["resource":"Amp\ByteStream\ResourceInputStream":private]=>
            resource(54) of type (stream)
            ["watcher":"Amp\ByteStream\ResourceInputStream":private]=>
            string(1) "b"
            ["deferred":"Amp\ByteStream\ResourceInputStream":private]=>
            &NULL
            ["readable":"Amp\ByteStream\ResourceInputStream":private]=>
            &bool(true)
          }
          ["error":"Amp\Process\ProcessInputStream":private]=>
          NULL
        }
        ["stderr"]=>
        object(Amp\Process\ProcessInputStream)#33 (5) {
          ["initialRead":"Amp\Process\ProcessInputStream":private]=>
          NULL
          ["shouldClose":"Amp\Process\ProcessInputStream":private]=>
          bool(false)
          ["referenced":"Amp\Process\ProcessInputStream":private]=>
          bool(false)
          ["resourceStream":"Amp\Process\ProcessInputStream":private]=>
          object(Amp\ByteStream\ResourceInputStream)#42 (4) {
            ["resource":"Amp\ByteStream\ResourceInputStream":private]=>
            resource(55) of type (stream)
            ["watcher":"Amp\ByteStream\ResourceInputStream":private]=>
            string(1) "c"
            ["deferred":"Amp\ByteStream\ResourceInputStream":private]=>
            &object(Amp\Deferred)#31 (4) {
              ["resolved":"Amp\Deferred":private]=>
              bool(false)
              ["result":"Amp\Deferred":private]=>
              NULL
              ["onResolved":"Amp\Deferred":private]=>
              object(Closure)#63 (2) {
                ["this"]=>
                object(Amp\Coroutine)#61 (9) {
                  ["generator":"Amp\Coroutine":private]=>
                  object(Generator)#60 (0) {
                  }
                  ["onResolve":"Amp\Coroutine":private]=>
                  *RECURSION*
                  ["immediate":"Amp\Coroutine":private]=>
                  bool(true)
                  ["exception":"Amp\Coroutine":private]=>
                  NULL
                  ["value":"Amp\Coroutine":private]=>
                  NULL
                  ["resolved":"Amp\Coroutine":private]=>
                  bool(false)
                  ["result":"Amp\Coroutine":private]=>
                  NULL
                  ["onResolved":"Amp\Coroutine":private]=>
                  object(Closure)#64 (2) {
                    ["this"]=>
                    object(Amp\Coroutine)#54 (9) {
                      ["generator":"Amp\Coroutine":private]=>
                      object(Generator)#53 (0) {
                      }
                      ["onResolve":"Amp\Coroutine":private]=>
                      *RECURSION*
                      ["immediate":"Amp\Coroutine":private]=>
                      bool(true)
                      ["exception":"Amp\Coroutine":private]=>
                      NULL
                      ["value":"Amp\Coroutine":private]=>
                      NULL
                      ["resolved":"Amp\Coroutine":private]=>
                      bool(false)
                      ["result":"Amp\Coroutine":private]=>
                      NULL
                      ["onResolved":"Amp\Coroutine":private]=>
                      object(Closure)#65 (1) {
                        ["parameter"]=>
                        array(1) {
                          ["$exception"]=>
                          string(10) ""
                        }
                      }
                      ["resolutionTrace":"Amp\Coroutine":private]=>
                      NULL
                    }
                    ["parameter"]=>
                    array(2) {
                      ["$exception"]=>
                      string(10) ""
                      ["$value"]=>
                      string(10) ""
                    }
                  }
                  ["resolutionTrace":"Amp\Coroutine":private]=>
                  NULL
                }
                ["parameter"]=>
                array(2) {
                  ["$exception"]=>
                  string(10) ""
                  ["$value"]=>
                  string(10) ""
                }
              }
              ["resolutionTrace":"Amp\Deferred":private]=>
              NULL
            }
            ["readable":"Amp\ByteStream\ResourceInputStream":private]=>
            &bool(true)
          }
          ["error":"Amp\Process\ProcessInputStream":private]=>
          NULL
        }
        ["pidDeferred"]=>
        object(Amp\Deferred)#24 (4) {
          ["resolved":"Amp\Deferred":private]=>
          bool(true)
          ["result":"Amp\Deferred":private]=>
          int(22458)
          ["onResolved":"Amp\Deferred":private]=>
          NULL
          ["resolutionTrace":"Amp\Deferred":private]=>
          NULL
        }
        ["status"]=>
        int(1)
        ["__propertySuggestThreshold":"Amp\Process\Internal\ProcessHandle":private]=>
        int(70)
      }
      ["value"]=>
      resource(56) of type (stream)
      ["__propertySuggestThreshold":"Amp\Loop\Watcher":private]=>
      int(70)
    }
    ["is_pending"]=>
    bool(false)
    ["priority"]=>
    int(0)
  }
}
kelunik commented 6 years ago

Does it work if you remove the reference from the foreach, so foreach ($this->events as $event) instead of foreach ($this->events as &$event)?

optiman commented 6 years ago

No, reference doesn't matter. The first array element $this->events['b'] simply is null.

kelunik commented 6 years ago

We have released Amp v2.0.5 which fixes this.