The current behaviour of the threaded worker is to immediately kill all the threadpools and force the reserved jobs to only reappear on the ready queues when they reach TTL.
This PR switches the approach to call #shutdown on each pool to prevent workers from reserving new jobs. Any workers already performing a blocking reserve (ie there's nothing in the tube) will mean the worker process won't die until the shutdown_timeout it reached (defaults to 10 seconds).
To make this work I had to switch up the signal handling for the threaded worker to use a self-pipe approach to escape the trap context (the same approach as sidekiq).
The graceful shutdown timeout period can be ended early by sending another TERM/INT signal to the process.
Also included in this PR are fixes to the threaded worker logging (timing code was not threadsafe) and a change to the test_helper load path (on my machine it was the installed gem that was being tested, rather than the repo). Let me know if you would rather I pull these out into their own PRs.
nesquena
commented
6 years ago
The current behaviour of the threaded worker is to immediately kill all the threadpools and force the reserved jobs to only reappear on the ready queues when they reach TTL.
This PR switches the approach to call
#shutdownon each pool to prevent workers from reserving new jobs. Any workers already performing a blocking reserve (ie there's nothing in the tube) will mean the worker process won't die until theshutdown_timeoutit reached (defaults to 10 seconds). To make this work I had to switch up the signal handling for the threaded worker to use a self-pipe approach to escape the trap context (the same approach as sidekiq). The graceful shutdown timeout period can be ended early by sending another TERM/INT signal to the process.Also included in this PR are fixes to the threaded worker logging (timing code was not threadsafe) and a change to the test_helper load path (on my machine it was the installed gem that was being tested, rather than the repo). Let me know if you would rather I pull these out into their own PRs.