michelsalib / BCCResqueBundle

The BCC resque bundle provides integration of php-resque to Symfony2. It is inspired from resque, a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.
122 stars 91 forks source link

How can I set a Job priority #143

Closed DavidGarciaCat closed 7 years ago

DavidGarciaCat commented 7 years ago

Imagine I have a process that queues some email alerts. Then, after these alerts have been queued, I get a new request and I need to queue a high priority task. By default, all Jobs are queued and processed as FIFO.

Is there a way to set-up priorities depending on the queued job, so we can specify what job needs to be processed first?

Thanks,

danhunsaker commented 7 years ago

The way Resque handles this is by creating multiple queues. Say, default, high, emergency, and low. When you start your worker(s), you can choose to dedicate workers to certain queues (such as one or two that only listen on emergency), or to list the queues in descending order of precedence (say, five workers listening to emergency,high,default,low). Resque will check each queue in order, and process the first job it finds, whatever queue it might be from. In your example scenario, your high priority job would be run by the first worker to finish any other job, and no low priority jobs would be touched until the default queue is empty.

Hopefully that makes sense.

DavidGarciaCat commented 7 years ago

Hello @danhunsaker

Yes, that makes sense, but it means I have one more question. Based on the example code that we can find in README.md file:

<?php

// get resque
$resque = $this->get('bcc_resque.resque');

// create your job
$job = new MyJob();
$job->args = array(
    'file'    => '/tmp/file',
    'content' => 'hello',
);

// enqueue your job
$resque->enqueue($job);

How can I specify the queue to use? I don't see it neither on this example nor on other examples (looking for default to find it).

Thanks for your help,

DavidGarciaCat commented 7 years ago

Is this the way?

<?php

namespace My;

use BCC\ResqueBundle\Job;

class MyJob extends Job
{
    public function __construct()
    {
        $this->queue = 'my_queue';
    }

    public function run($args)
    {
        ...
    }
}
nickdtodd commented 7 years ago
<?php

// get resque
$resque = $this->get('bcc_resque.resque');

// create your job
$job = new MyJob();
$job->queue = 'queue_name';
...

For your first example.

danhunsaker commented 7 years ago

If you know a job will always be placed on a certain queue, you can declare that in the class definition itself, but otherwise, the above is your best bet.

DavidGarciaCat commented 7 years ago

Hi @nickdtodd and @danhunsaker

Thanks for your replies. I will try to test this.

Enjoy the rest of your day!

DavidGarciaCat commented 7 years ago

Tested and working - many thanks guys!