Wait until all jobs complete?

[amro]

@brandonhilkert, thanks for the gem. It's great.

I'm using whenever gem to start a bunch of sucker_punch jobs via cron. I want to the jobs async so I can run >1 at a time.

My problem is this: After creating all of the jobs, the method that creates the jobs returns and so the process dies and that means the jobs don't get executed.

My question: Is there a way to prevent the main thread from terminating until all jobs are complete? Put another way, is it possible to query for a count of outstanding sucker_punch jobs? (I could then do something like use a while loop that sleeps for 1/10 of a second, wakes up, and queries the number of outstanding jobs, then breaks out when all jobs are complete.)

Thanks in advance.

P.S. I could either write or use a long lived daemon for this but I'm trying to avoid the complexity of monitoring the daemon.

Edit: And I definitely made an assumption here about why jobs are failing. I could be mistaken and I could be doing something else wrong entirely.

[brandonhilkert]

You could try a counter like object like http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/CountDownLatch.html. Currently, SP doesn't have any stats. I've been working on version based on concurrent-ruby, which will have all of that. You can follow allow on the concurrent branch if you're interested. Shouldn't be too much longer.

[amro]

Thanks!

Thanks, Amro

Sent from my iPhone

On Nov 18, 2015, at 5:22 PM, Brandon Hilkert notifications@github.com wrote:

Closed #139.

— Reply to this email directly or view it on GitHub.

[natanaelmaia]

Amro, you can also rely on Celluloid futures, which sucker_punch is based on. Instead of calling job.async.perform method, try call job.future.perform (https://github.com/celluloid/celluloid/wiki/futures).

ex:

def some_method
    futures = []
    job_data.each do |some_job_data|
        futures << SomeJob.new.future.perform(some_job_data)
    end
    futures.each{ |future| future.value } 
end

Each ".value" call will block and some_method will return only after all jobs are finished.

[amro]

Thanks @natanaelmaia! That's even better.

Edit: Both worked great - ended up going with the second solution since it's a bit simpler and I don't need to pass around the semaphore. TYVM.