search

dereuromark / cakephp-queue

Queue plugin for CakePHP - simple, pure PHP and without dependencies.
MIT License
306 stars 137 forks source link

Parallel tasks #335

Closed DanielMescoloto closed 2 years ago

DanielMescoloto commented 2 years ago

Hey Mark Amazing plugin by the way

But I am having a problem with multiple workers getting the same task at the exact same moment. I even add$unique = true; on every task I add a reference name for each task as well

the database is SQL Server (Microsoft) I have 3 workers running at the same time

'sleeptime' => 3, 'gcprob' => 10, 'defaultworkertimeout' => 1800, 'defaultworkerretries' => 3, 'workermaxruntime' => 298, // 15 minutes 'workertimeout' => 120 * 100, 'exitwhennothingtodo' => false, 'cleanuptimeout' => 2592000, // 30 days 'maxworkers' => 5, 'multiserver' => true, 'connection' => 'default',

dereuromark commented 2 years ago

That might be the issue. Use a transaction safe DB as then each run will make sure to be atomic in the query and save. Then the next run would definitly not be able to take this very same task. Mysql InnoDB or Postgres does this afaik.