Would Gearman be an appropriate dependency to add?

[itsjohncs]

This is an alternative to Celery and might be better suited for the task than Celery according to some random people online. I'm guessing that they're speaking from a mostly performance perspective, which we don't really care about because the amount of messages passed around Galah is absolutely tiny compared to the benchmarks these solutions are spouting.

I'd really like to focus on how code using Gearman would look compared to code using Celery and how maintainable it would end up being. Stability and fault tolerance is another thing that will be very important. The information provided to system administrators will also be important.

[itsjohncs]

A presentation here concludes by saying that Gearman is too young at the moment. Given that it's a five year old project this is surprising. It also says that the queues are unstable and start misbehaving if there are too many items in the queue. This is very upsetting and seems to validate my worry that the fact that Gearman implements its own queuing system will lead to problems. The presentation actually says Gearman has all the problems I didn't want it to have. I need much more evidence besides just a slideshare presentation I found randomly online, but this is pretty damming if it holds up.

[itsjohncs]

Yelp maintains the Gearman Python API. Bonus points there, but looking into the development history and issues does not give me a happy feeling. Very little activity... The yelp team also seems to be fairly unresponsive.

This touches on a larger problem I have with Gearman is that the project is fairly fragmented and we'd be dealing with different teams depending on what's going on. I'm also not finding too many resources for developing with Gearman in Python.

[itsjohncs]

After researching Gearman for some time today it doesn't seem like I'll find a lot of support for using it with Python. The fact that is has its own daemon process that doesn't receive anywhere near the attention that Redis does, and rumors of instability and explosions makes me worry. The project is not hosted on GitHub which makes it more difficult for me to keep a pulse on it (it also uses Bazaar as the versioning system which is not something I'm at all familiar with).

My conclusion is that despite Gearman's excellent pedigree and despite the fact that it is a great contender, I don't think the complexity it will add to Galah will be worth the benefits it brings to the table. I am now moving on to investigate the costs and benefits of using Celery vs using Redis directly.