collectiveidea / delayed_job

Database based asynchronous priority queue system -- Extracted from Shopify
http://groups.google.com/group/delayed_job
MIT License
4.81k stars 955 forks source link

Suggestion: Rake task to lauch multiple monitored workers in a container context #1189

Open svoop opened 1 year ago

svoop commented 1 year ago

There are a few issues concerning Delayed::Job and Docker. I'm migrating a Rails app with Delayed::Job to render.com and couldn't do out of the box.

(I have to admit that I found running multiple monitored workers to be quite a challenge in almost every situation. For example, delayed_job -n 5 -m start does daemonize on macOS, but works fine on Linux.)

Here's a Rake task I came up with and it appears to work pretty okay so far:

https://gist.github.com/svoop/12de6e3f7a447dc0e036ce1ca35d11ba

There's some coupling with Rails so it's certainly not suitable to be added to Delayed::Job as is, but it might be an inspiration for a slightly different workmulti task.

The task doesn't return control but becomes the monitor. It's therefore working just fine to RUN with Docker or in any similar context like my deploys to render.com.