search

brandonhilkert / sucker_punch

Sucker Punch is a Ruby asynchronous processing library using concurrent-ruby, heavily influenced by Sidekiq and girl_friday.
MIT License
2.64k stars 114 forks source link

How max_jobs is supposed to work? #242

Closed glaucocustodio closed 3 years ago

glaucocustodio commented 3 years ago

I have this job:

class MyJob
  include SuckerPunch::Job
  workers 1
  max_jobs 1

  def perform(foo)
  end
end

If I open more than one rails console I can trigger it multiple times with MyJob.perform_async(foo) and don't see error Concurrent::RejectedExecutionError. Even running 2.times { MyJob.perform_async(foo) } I don't see the error.

I need to make sure this job is run only once at the time.

brandonhilkert commented 3 years ago

Is the job doing anything? Are you sure it's not completing faster than the jobs are being enqueued?

I made a quick script with the following:

require 'sucker_punch'

class MyJob
  include SuckerPunch::Job
  workers 1
  max_jobs 1

  def perform
    sleep 2
  end
end

5.times { MyJob.perform_async }

It produces the following:

[07:45:05] brandonhilkert [~/code/sucker_punch] (master) $ ruby ~/Desktop/sp.rb
I, [2021-03-27T19:45:08.706620 #71807]  INFO -- : Pausing to allow workers to finish...
Traceback (most recent call last):
    15: from /Users/brandonhilkert/Desktop/sp.rb:13:in `<main>'
    14: from /Users/brandonhilkert/Desktop/sp.rb:13:in `times'
    13: from /Users/brandonhilkert/Desktop/sp.rb:13:in `block in <main>'
    12: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sucker_punch-3.0.1/lib/sucker_punch/job.rb:38:in `perform_async'
    11: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sucker_punch-3.0.1/lib/sucker_punch/queue.rb:169:in `post'
    10: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:41:in `synchronize'
     9: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:41:in `synchronize'
     8: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:41:in `block in synchronize'
     7: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/sucker_punch-3.0.1/lib/sucker_punch/queue.rb:171:in `block in post'
     6: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/ruby_executor_service.rb:19:in `post'
     5: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:41:in `synchronize'
     4: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:41:in `synchronize'
     3: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:41:in `block in synchronize'
     2: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/ruby_executor_service.rb:22:in `block in post'
     1: from /Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:159:in `ns_execute'
/Users/brandonhilkert/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/abstract_executor_service.rb:87:in `handle_fallback': Concurrent::RejectedExecutionError (Concurrent::RejectedExecutionError)