Memory issues

[maciejfikus]

Hello, that's me again :-)

I've played with mysql driver for a little, and there is some behavior that I am not sure how to interpret. Same thing happen on PHP 8.1 and 8.2 and also with 0.5 version of package as well as 0.6-dev.

$connection = (new Factory())->createLazyConnection(
    'devuser:devpass@172.29.0.1:3307/devdb'
);

for ($i = 0; $i <= 30000; $i++) {
    $connection->query('SELECT 2')->then(
        fn(QueryResult $result) => 5,
        fn(Exception $ex) => 1
    );
}

Loop::addPeriodicTimer(3, async(static function () use ($connection) {
    var_dump(getMemory(), $connection->pending);
}));

Loop::addPeriodicTimer(10, async(static function () {
    var_dump([
        'gc_collect_cycles' => gc_collect_cycles(),
        'gc_mem_caches' => gc_mem_caches()
    ]);
}));

In the code above, I simply created a DB connection and tried to spawn many queries in "fire and forget" style. Every 3 seconds, I'm checking memory consumption alongside with pending queries counter.

Memory dump as soon as queries finish:

string(22) "Used Memory : 57.33 MB"
string(25) "Used Real Memory : 706 MB"
string(30) "Used Real Peak Memory : 706 MB"

Memory dump after gc_collect_cycles and gc_mem_caches are fired:

string(22) "Used Memory : 9.06 MB"
string(25) "Used Real Memory : 604 MB"
string(30) "Used Real Peak Memory : 706 MB"

Used memory is fine, but reserved memory (used real memory) won't be freed. I can fire another 30k queries, and it will work within already reserved memory, but running i.e. 40k queries will increase reserved memory infinitely and so on. I've a feeling that there is some pointer on the loop which keep references to fibers, but I've no idea how to validate that idea.

I even tried to unset whole connection variable and set it to null, but it didn't change a thing :(

In simple words, I don't really understand why, would be very grateful for an explanation.

[SimonFrings]

Hey @maciejfikus, thanks for bringing this up :+1:

Based on my knowledge, PHP requires a certain amount of memory from your system and the amount depends on how quickly the used memory increases. Your system tries to be clever by predicting how much memory you'll need, depending on how aggressively you're using it. Even after an operation finishes, the reserved memory doesn't immediately drop in case it's needed again at a later time. It's beneficial to avoid constant communication between PHP and your system to determine the required memory, as this saves time and improves overall performance. It's difficult for me to estimate the specific amount of memory your system would need, so I can't confirm if these numbers are accurate or not.

[…] but running i.e. 40k queries will increase reserved memory infinitely and so on.

Using 40k queries might make the system allocate even more memory to PHP as your operations might surpass the reserved memory. I can't really see the memory growing infinite in your examples above, but I could understand the behavior of giving out more memory for more operations.

We're currently not aware of any memory leaks in this project and taken from your input above, I'm not quite convinced that this could be a memory issue related to ReactPHP. I'll go ahead and close this ticket for now, but we can reopen this if you can proof that ReactPHP or the friends-of-reactphp/mysql project is causing memory issues. If you do need to reopen the ticket, please provide some information about the steps you've taken so far, what worked and what didn't.