Ruby process quits before Async jobs finish

[jjbooth74]

Similar to #203 (feel free to merge with that if it makes sense).

When queuing an async job from within another async job, the process quits before the inner jobs are allowed to run.

Bundler.require

class OutsideJob
  include SuckerPunch::Job
  workers 2

  def perform(args = {})
    puts 'Outside job'
    InsideJob.perform_async(args[:monkeys])
  end
end

class InsideJob
  include SuckerPunch::Job
  workers 10

  def perform(monkeys)
    puts "#{monkeys} monkeys"
  end
end

puts 'Performing jobs'
OutsideJob.perform_async(monkeys: ARGV[0])

$ bundle exec ruby script.rb 123
Performing jobs
Outside job

I would expect the process to wait until all jobs are completed, not just the outside job.

If I switch the InsideJob to InsideJob.new.perform(...) then I get:

$ bundle exec ruby script.rb 123
Performing jobs
Outside job
123 monkeys

[brandonhilkert]

When the script exits, the at_exit handler is called. As part of the shutdown process, all queues are blocked, not allowing new jobs to be enqueued. So in this case, the OutsideJob is enqueued and the script exits shutting down all queues before the code in OutsideJob can enqueue the InsideJob, which is never does b/c the queues do not allow it.

There would be no way to reliably do this without interjecting an arbitrary amount of sleep time assuming new jobs would be enqueued by then, which would be unreasonable b/c there's no telling how many OutsideJob jobs are enqueued, the number of workers set, and then to consider what the code in OutsideJob is doing and how long it will take.

In this case, I would compose the methods from InsideJob and run them inline in OutsideJob.