Priority function?

[dennisoderwald]

Hi,

can i give a job a priority? :)

[wa0x6e]

No.

But you can give queues priority

[dennisoderwald]

How?

Can i start 3 Queues with overall 5 workers for all queues?

[wa0x6e]

First, you start workers, not queues. Each workers can poll one or more queues.

You can set 3 queues for each priority : low, medium, high.
Then you push your job into the queues, depending on their priority.

Each queue must be polled by at least one worker, so you should have 3 workers minimum in that case, one for each queue.

• Worker A => high
• Worker B => medium
• Worker C => low

You're free to add more workers, like having a worker D and E for the high queue:

• Worker A => high
• Worker D => high
• Worker E => high
• Worker B => medium
• Worker C => low

That way, your high priority jobs are executed more faster, and independently of the other priority.

There is another definition of priority, that is high priority come before low priority, so when there a high priority job, execute it first, then execute the lower priority jobs. In that case, you need only one worker for all queues:

• Worker A => high, medium, low

Just start your worker with start -q "high,medium,low". That worker will poll the 3 queues, in the definition order.

[dennisoderwald]

high medium or low is the queue Name? Or a optional flag in the Configuration?

[wa0x6e]

queue name

[dennisoderwald]

Thanks!

[dennisoderwald]

Is the name of the queue for the priority or the order in the start -q command?

Because I need more than 3 priorities..

[wa0x6e]

-q takes a list of queues. When defining multiple queues, like "queue1,queue2", they are polled in the definition order, so queue1 will always be polled before queue2.

[dennisoderwald]

I must start more queues on a worker with the cli or can define this in the configuration and i use 'start' - all options are loaded and maybe: 5 workers with 3 queues and 1 worker with 1 queue start..? :) Actually i use a own start bash script.. but in the configuration were nice..

[wa0x6e]

You can define your workers settings in the [Queues] section of the config file, then load them using load

[dennisoderwald]

You mean..

        'Queues' => array(
            array(
                'queue' => 'high',  // Use default values from above for missing interval and count indexes
                'user' => 'www-data'    // If PHP is running as a different user on you webserver
            ),
            array(
                'queue' => 'medium',
                'user' => 'www-data'
            ),
            array(
                'queue' => 'low',
                'user' => 'www-data'
            )
        ),

or? But i will 3 workers with high,medium and 1 worker with "low"

[wa0x6e]

'Queues' => array(
            array(
                'queue' => 'high,medium',
                'workers' => 3
            ),
            array(
                'queue' => 'low',
            )
        ),

[dennisoderwald]

Ah thanks kamisama, had the option misunderstood..

[dennisoderwald]

Hm.. only starts 5 workers with the default queue, defined as 'Worker'

        'Worker' => array(
            'queue' => 'medium',        // Name of the default queue
            'interval' => 1,            // Number of second between each poll
            'workers' => 5,             // Number of workers to create
            'user' => 'www-data' ,  // User running the worker process

            // Path to the log file
            // Can be an
            // - absolute path,
            // - an relative path, that will be relative to
            //   app/tmp/logs folder
            // - a simple filename, file will be created inside app/tmp/logs
            'log' => TMP . 'logs' . DS . 'resque-worker-error.log',

            // Log Verbose mode
            // true to log more debugging informations
            // Can also be enabled per worker, by starting with --verbose
            // 'verbose' => true
        ),

Queues defined as your example..

[wa0x6e]

you're starting them with load, aren't you ?

[dennisoderwald]

Ah thx! :)