spatie / async

Easily run code asynchronously
https://spatie.be/en/opensource/php
MIT License
2.65k stars 179 forks source link

Added a memoryEfficient mode #219

Closed SRWieZ closed 7 months ago

SRWieZ commented 11 months ago

Added a memoryEfficient() mode, when true, it omit storing $this->results and $this->finished after processing tasks. I think this mode can be beneficial since in many case, we don't need to recheck out the result afterwards.

Use case I used your package to compute a lot of calculation with large range of numbers. So big that I had to use a php generator. After setting the pool I used htop to look at memory usage and it was going up and up, until hitting memory limit (more than 5Go !) After my PR, the memory usage stay stable.

Example usage

function rangeGenerator($start, $end)
{
    for ($i = $start; $i <= $end; $i++) {
        yield $I;
    }
}

$pool = Pool::create()
    ->memoryEfficient();
    ->concurrency(32);

$find_min = null;
foreach (rangeGenerator(0, 10000000) as $seed) {
  $pool
    ->add(function () use ($seed) {
      // Calculation related to $seed

      return $result;
    })
    ->then(function ($output) use (&$find_min) {
      if (is_null($find_min) || $output < $find_min) {
        $find_min = $output;
        echo "New min : " . $find_min . "\n";
      }
    });
}
echo "Result : " . $find_min . "\n";

Side note: Should we throw an exception on getFinished() if memoryEfficient is set to true ?

spatie-bot commented 7 months ago

Dear contributor,

because this pull request seems to be inactive for quite some time now, I've automatically closed it. If you feel this pull request deserves some attention from my human colleagues feel free to reopen it.