nesquena / backburner

Simple and reliable beanstalkd job queue for ruby
http://nesquena.github.com/backburner
MIT License
428 stars 68 forks source link

Added ability to handle all calls to an instance or class method asynchronously #100

Closed contentfree closed 9 years ago

contentfree commented 9 years ago

Allows the developer to configure async usage separately from their code. This is especially useful when you only want asynchronous behavior in a particular environment. Inspired heavily by the same concept in Delayed::Job

Example:

class Example
  def id: 1 end
  def hard_task; "…work…" end
end

# In a production config, perhaps:
Backburner::Performable.handle_asynchronously(Example, :hard_task)

# Now, this will use #async automagically:
Example.new.hard_task

There's also a way to handle class methods:

class Example
  def self.hard_task; "…work…" end
end

# Elsewhere:
Backburner::Performable.handle_static_asynchronously(Example, :hard_task)

# It now uses #async
Example.hard_task

NB: If the producer and consumer are not configured the same way – eg. the producer uses #handle_asynchronously in its setup but the consumer doesn't – then the consumer will need to know that the method stored in beanstalkd has a suffix of "_without_async". (In the first example above, Backburner will actually store the method name as :hard_task_without_async.) This seems like a rare edge case, but it's imaginable.

contentfree commented 9 years ago

Another note: This only works on persisted objects aka things with an id and a find(id) method. I'm doing some tests to see if a normally-transient instance of a class will work if given an id method that returns an object and a find(id = {}) method that rebuilds the transient instance.

If that works, then it might be nice to either be able to either provide serialize and finder methods when calling handle_asynchronously (to preserve the "I'm setting up async outside the knowledge of my class" aspect) or to formalize it in Performable to remove the assumptions of id and find(id).

nesquena commented 9 years ago

Can you add a description of how this works to the README? I would like to merge this in and cut a new release soon.

contentfree commented 9 years ago

Added an example to the README and rebased to the current master

nesquena commented 9 years ago

Why not make this work similar to delayed jobs where you just call it directly on the object:

https://github.com/collectiveidea/delayed_job#queuing-jobs

Am I misunderstanding or why this:

Backburner::Performable.handle_asynchronously(User, :activate, ttr: 100, queue: 'activate')

instead of

class User
  include Backburner::Performable
  handle_asynchronously(:activate, ttr: 100, queue: 'activate')
end

?

contentfree commented 9 years ago

The former is cleaner inasmuch that you don't have to reopen the class to add the module and call a newly-added static method.

I like to add async behaviour in my production config (which is what I did with Delayed Job, too, with its handle_asynchronously method... It just sullied up the object space, I think)

In the end, it doesn't matter that much - I could write a helper to do the reopen and include, etc - but this seems like better separation.

I do it this way for 6-8 methods in my set of models and six lines of this looks tidier than reopening the same number of classes and doing the same thing to each.

/shrug

nesquena commented 9 years ago

In my view, calling a class method on a module feels awkward and its unclear where you would call that line. This resque implementation https://gist.github.com/jdhollis/432220 also works within the class as does the delayed_job implementation. I prefer consistency with what people have used unless it's a significant improvement.

nesquena commented 9 years ago

Alternatively we could keep this out of core such similar to https://github.com/nesquena/backburner_mailer. Resque never added this into core AFAIK. Does sidekiq have an equivalent?

contentfree commented 9 years ago

I have most of my handle_asynchronously calls in a configure :production block (depending on the web framework…) so using a module function was almost identical to my usage of DJ's handle_asynchronously. (This style is a bit more flexible that DJ's since you can use it on class methods, too.)

However, I see no reason why the Performable module can't also include handle_(static_)asynchronously when included and be used as you suggest. (I just prefer keeping the knowledge of async out of my models.)

"Why not both?"

contentfree commented 9 years ago

@nesquena Now you can call #handle_asynchronously on the class that included the Performable module. Uses the same internals as my original commit. So now we can both be happy ;)

contentfree commented 9 years ago

Updated the readme, too.

nesquena commented 9 years ago

Great, glad it was an easy addition to add the additional syntax. Will merge in once travis finishes.

contentfree commented 9 years ago

@nesquena: Travis seems stuck

nesquena commented 9 years ago

It does :(

nesquena commented 9 years ago

Passed and merged!