Yelp / python-gearman

Gearman API - Client, worker, and admin client interfaces
http://github.com/Yelp/python-gearman/
Other
242 stars 124 forks source link

Retry gracefully when a server restarts #42

Closed hashbrowncipher closed 11 years ago

hashbrowncipher commented 11 years ago

Currently, python-gearman fails to correctly retry submitting jobs when a connection it holds to the gearmand 'flakes.' A 'flake' of a gearmand is when the TCP connection is temporarily broken, but the gearmand is listening as of the next retry. Commonly we see this when gearmand is restarted, as the previous TCP connection between client and server will no longer be open.

Currently, poll_connections_until_stopped will stop polling on a Connection object which has become disconnected. This is a mistake, because the Connection object may become reconnected by virtue of send_job_request, which is called recursively by the callback function continue_while_jobs_pending. In that case, the client should definitely be polling on that Connection object, because it is both connected and has valid data to send to the server.

I also added a comment on an unrelated function and fixed a few pyflakes issues that I saw.