Does Faktory have any notion of job cost?

[knpwrs]

Say I want to assign a cost value to each job in a queue, and say I have a worker node that is capable of processing jobs adding up to 11,520 cost points. This worker node can process three jobs worth 3840 points, or two jobs worth 3840 and two jobs worth 1920 points, and so on.

Is there any way to model this in Faktory? The way I envision it working is say the worker node I mentioned is currently running two 3840 jobs and one 1920 job for a total of 9,600. If the next job in the queue is 3840, but the job after is only 1920, then the worker should take the 1920 job, even though the 3840 job was placed in the queue first.

[mperham]

That's equivalent to job priority. Faktory does not support that, as it would change the queue PUSH/POP from O(1) to O(log N), impacting scalability pretty severely.

[knpwrs]

I see. The go and node worker libraries seem to have the notion of switching between priority queues, but they also cap at an overall number of jobs rather than adapting to capacity on the worker. Maybe this concept could be implemented on the worker side, where each queue is associated with a certain cost and the worker will run in parallel up to a certain total cost?

[mperham]

The Go library has a notion of queue weights and so jobs in a higher priority queue will take precedence. Jobs themselves have no intrinsic cost or priority.