bkeepers / qu

a Ruby library for queuing and processing background jobs.
MIT License
505 stars 50 forks source link

Optionally fork before running each job #5

Closed bkeepers closed 10 years ago

bkeepers commented 12 years ago

This should be an option that is easy to turn on or off.

johnae commented 12 years ago

Or threads for jruby?

bkeepers commented 12 years ago

Sure, I'm not opposed to that.

DeTeam commented 11 years ago

Possible configuration option:

Qu.process_with = lambda do |job_closure|
  Thread.new { job_closure.call }
end

Qu.process_with = lambda do |job_closure|
  job_closure.call if fork
end

By default it's just .call on arg. What do you think?

bkeepers commented 11 years ago

I think I would rather see it wrapped up in a class so it can be tested and extended. I have a hunch the different strategies will be more complicated than you'd want to handle in a lambda.

class ThreadedProcessor
  def initializer(max_threads = 5)
    @max_threads = max_threads
  end

  def run(job)
    @threads << Thread.new { job.process }
    wait_until_a_thread_dies if at_max_threads?
  end

  # …
end

Qu.job_processor = ThreadedProcessor.new(10)
DeTeam commented 11 years ago

Agree, strategy classes seem fine.

There's also an idea to move backend interface to a separate class with "implement me" warning.

jnunemaker commented 10 years ago

This is done. There is a runner that can be customized in #86