wa0x6e / Cake-Resque

Resque plugin for CakePHP : for creating background jobs that can be processed offline later
MIT License
159 stars 56 forks source link

Starting workers in parallel may fail due to same pidFile #106

Closed mfn closed 2 years ago

mfn commented 7 years ago

When having lots of queues and lots of workers, the time it takes to serially start all workers gets noticeable. In my case this is >30s.

When trying to start them in parallel with e.g. cake cake_resque -q foo -n 5 & we run into troubles that sometimes a worker is reported as Fail during startup.

We traced this to the following code-line https://github.com/wa0x6e/Cake-Resque/blob/42c4fbcca3cd8e5ae7817604887e84208552f1d2/src/Shell/CakeResqueShell.php#L825 :

$pidFile = Configure::read('CakeResque.Resque.tmpdir') . str_replace('.', '', microtime(true));

It's unlikely but not impossible that two workers receive the same value from microtime(true). The entropy is simply too low.

This $pidFile gets deleted after a worker successfully starts so although both workers start correctly up, one of them is faster deleting this file and the other is thus reported as being false (and: likely still to run correctly).

mfn commented 2 years ago

Closing, no longer using CakeResque