pmmp / ext-pmmpthread

Fork of https://github.com/krakjoe/pthreads with a revamped API and PHP 8.1+ support
Other
81 stars 17 forks source link

Closure reuse memory leak #103

Closed dktapps closed 1 year ago

dktapps commented 1 year ago

Because of this code: https://github.com/pmmp/pthreads/blob/403894bba8774f131df8bd8cd3dfce20617028d6/src/store.c#L1079

and this code: https://github.com/pmmp/pthreads/blob/403894bba8774f131df8bd8cd3dfce20617028d6/src/copy.c#L705

the op_array for closures is never reused, but lives forever on the destination thread in EG(function_table), which causes memory usage to spiral on the child thread like crazy in cases like the following:

$channel = new \ThreadedArray();

$thread = new class($channel) extends \Thread{
    public bool $shutdown = false;

    public function __construct(
        private \ThreadedArray $channel
    ){}

    public function run() : void{
        $i = 0;
        ini_set('memory_limit', '-1');
        while(!$this->shutdown){
            $closure = $this->channel->synchronized(function() : \Closure{
                while($this->channel->count() === 0){
                    $this->channel->wait();
                }
                return $this->channel->shift();
            });
            $closure($i);
            if(($i % 100000) === 0){
                var_dump("child thread: " . number_format(memory_get_usage()));
                usleep(1000);
            }
        }
    }
};

$thread->start();
for($i = 0; ; $i++){
    $channel->synchronized(function() use ($channel) : void{
        $channel[] = function(int &$v) : void{
            $v++;
        };
        $channel->notify();
    });
    if(($i % 100000) === 0){
        var_dump("parent thread: " . number_format(memory_get_usage()));
        usleep(1000);
    }
;
}
$thread->shutdown = true;
$thread->join();

You can see that while the parent thread's memory usage remains relatively stable, the child thread memory usage skyrockets.