yejunzhou / typhoonae

Automatically exported from code.google.com/p/typhoonae
0 stars 0 forks source link

Tasks can jam in the celery backend #76

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Create two different queues,

- name: fast
  rate: 10/s

- name: slow
  rate: 1/d

Enqueue several tasks in the "slow" queue, and then a task in the "fast" queue. 
The task in the "fast" queue will have to wait until all tasks in the "slow" 
queue are handled.

We need to assign each task class to a different celery queue to prevent this. 
I'm working on this.

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?

Please use labels and text to provide additional information.

Original issue reported on code.google.com by e98cu...@gmail.com on 15 Oct 2010 at 7:25

GoogleCodeExporter commented 9 years ago
This is fixed in trunk. Celery fetches tasks from its queues using round robin. 
A slow task queue will make other queues slower, but it will not stall them.

Users can run different celery workers for the very slow queues if they want to 
remove completely their effect on fast queues.

Original comment by e98cu...@gmail.com on 27 Oct 2010 at 8:07

GoogleCodeExporter commented 9 years ago
Fix included in 0.2.0 release.

Original comment by tobias.r...@gmail.com on 1 Jan 2011 at 11:28