Celery and other async job queues

[abhaga]

Celery and rq are mentioned in Appendix A but not discussed in the text. It would be good to have some discussion on best practices for what kind of tasks it is appropriate for, where should it sit in the project layout (Chapter 3) etc.

[pydanny]

Items:

1. Do you need a queue?
2. Queue options
3. Only pass primitives to tasks
4. Ignore results that you don't want
5. Use the queue's failure handling
6. Learn the quirks of your queue
7. Don't use database as the queue broker if you care about performance
8. Periodically clear out dead queue items (sometimes stuff just gets stuck in there)
9. Chain smaller tasks together rather than doing one giant task. Make sure you know exactly what you are doing when chaining
10. If debugging gets hard, move the business logic to normal functions that the async tasks call.
11. Logging! Task specific logging, statsd, etc
12. David Gouldin's presentation

[pydanny]

:ship: