search

contribsys / faktory_worker_ruby

Faktory worker for Ruby
GNU Lesser General Public License v3.0
214 stars 31 forks source link

How to determine that worker process is alive and working? #47

Closed Envek closed 4 years ago

Envek commented 4 years ago

Is there any way to check each worker process for its health? To be able to restart them in case if they will get stuck or freeze or anything else.

If worker isn't busy and isn't able to take new jobs, I want to kill it and restart.

It is primarily needed for deploying to Kubernetes as it is considered a best practice to always include liveness checks for containers in k8s pods. For example it may be any executable (e.g. shell script) that will return 0 if worker is healthy and something else if it is not. Also checks may be TCP or HTTP probes but I don't think that they fit for worker processes case.

Thank you for faktory!

scottrobertson commented 4 years ago

Have you experienced workers getting stuck, or unable to accept jobs? We are processing ~10m jobs per day, and this is not something we have seen.

Envek commented 4 years ago

We're not in production yet, so no, have not experienced… yet :smile:

mperham commented 4 years ago

https://github.com/mperham/sidekiq/wiki/Related-Projects#kubernetes

Envek commented 4 years ago

Ok, will look on how to re-implement sidekiq_alive for faktory.

Thank you!

mperham commented 4 years ago

Oh, sorry I didn’t notice this was FWR and not Sidekiq. Yep, the same principle applies. If you find it relatively easy and send me a PR we could probably bake this into the core gem as Docker and k8s seem to be more and more popular.

On Apr 3, 2020, at 09:36, Andrey Novikov notifications@github.com wrote:

 Ok, will look on how to re-implement sidekiq_alive for faktory.

Thank you!

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.