shinyscorpion / task_bunny

TaskBunny is a background processing application written in Elixir and uses RabbitMQ as a messaging backend
MIT License
202 stars 30 forks source link

response queue abstraction #48

Open norbu09 opened 6 years ago

norbu09 commented 6 years ago

I really like this framework and contemplate using it over my current abstractions for RabbitMQ. One pattern I use quite often is to spin up a response queue, do a few things in various workers and then send the response back to the response queue.

Before I start writing any code on that I wanted to know if you have any thoughts on where that code would best sit in TaskBunny, if it even sits there at all or if you have any preferred way for adding it so that I can play around with some code that has a chance of being merged back in.

ono commented 6 years ago

Hi @norbu09. I'm glad to hear you like the library.

I might not fully understand what you mean by the response queue but it doesn't sound like a scope of TaskBunny. TaskBunny is not a general RabbitMQ wrapper. TaskBunny is just a background processing library like Resque, Sidekiq, RQ etc. and we are bit cautious on adding extra features.

What you describe sounds like other type of message broker framework and maybe it doesn't make sense to add to TaskBunny?

norbu09 commented 6 years ago

I have started to implement one part of what I am planning to do here: https://github.com/shinyscorpion/task_bunny/pull/49

The new enqueue call would look something like this:

TaskBunny.Job.enqueue!(Worker.Store, %{"job" => %{"foo" => "bar", "two" => 1}}, reply_to: "Worker.Log")

This would send a message with {"ok" => result} to a Worker.Log worker for further processing.

I hope the code and my explanations make it more obvious what I try to do. If you don't see a good fit for this in the core framework then I will just continue to work on my fork. without a way to push messages through queue stages I can't really build the systems I normally build :)