brandonhilkert / sucker_punch

Sucker Punch is a Ruby asynchronous processing library using concurrent-ruby, heavily influenced by Sidekiq and girl_friday.
MIT License
2.65k stars 114 forks source link

Added perform_through to allow pushing different jobs to a named queue #246

Closed tarqmamdouh closed 2 years ago

tarqmamdouh commented 2 years ago

Defined Problem

I've been working lately on a project which heavily depends on asynchronous background jobs, I was facing a case where I needed to have more control on jobs queue.

Currently the gem finds or creates a queue based on the job class name, for my case some jobs must be queued under certain scopes so 2 different jobs with the same scope cannot be running at the same time.

Suggested Solution

Through reading the code of this gem, I found that we could have a new functionality that allows us to push a job to a new or an already named queue.

This will allow me to have more vision and control of when and where async jobs gets executed.

Changes Introduced

The main Idea is to add a function called perform_through which take a first argument of the queue options as described below: -

class DataJob
  include SuckerPunch::Job

  def perform(data)
    puts data
  end
end

options = {
  queue: 'data_queue', # default is class name
  workers: 2, # default is 1
  max_jobs: 10 # default is Unlimited
}
DataJob.perform_through(options, "asdf") # Job will be queued and `perform` will be executed when worker picks it up.

If the queue is not defined it will be created with given options. Otherwise, it will just push the job to it.

brandonhilkert commented 2 years ago

Thanks for your interest and proposal. Sucker Punch was intended to be a lightweight solution to async job management and I'd like to keep the API as-is. If you're in need of a more complex system, I'd recommend something like Sidekiq that's more robust.

Thanks again for your interest and putting forward the work. I enjoyed reading the implementation.