benmanns / goworker

goworker is a Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers.
https://www.goworker.org
Other
2.79k stars 241 forks source link

How does goworker work? #64

Closed basiszwo closed 6 years ago

basiszwo commented 6 years ago

I came across this project and I am wondering how it manages to execute arbitrary ruby code.

As from the ready I assume that I can have goworker run on my servers and from my ruby application I create some Resque-style background jobs which get processed by goworker.

I dig the code but I cannot find the spot where the ruby code gets executed and how you manage to get the correct ruby environment ...

Maybe I got something wrong though. Would be great if someone could shed some light on this?

Maybe @benmanns, @johnTurknett or @rjrobinson?

Thank you very much!

mingan commented 6 years ago

No Ruby gets executed. If you want the processing code to be in Ruby, you don't need goworker. If you, however, need to offload some jobs to Go, you can use goworker to manage a pool workers, assign them jobs from them the queue and bind Go functions to handle the workload.

The only relation to Ruby is that goworker emulates the behaviour of Resque with respect to how the queue is managed (Redis keys used, meaning of certain fields). The actual code processing the jobs is in Go.

basiszwo commented 6 years ago

ahh, ok. Now I get it. I was confused by the ruby example in the readme that was used to explain testing

class MyClass
  @queue = :myqueue
end

100.times do
  Resque.enqueue MyClass, ['hi', 'there']
end

Now it's clear to me. 👍

mingan commented 6 years ago

jj, the typical use case is a Ruby application using Resque gem to enqueue jobs and goworker running with the code that consumes the jobs. If you are fine with processing code in Ruby, you don't need goworker.

basiszwo commented 6 years ago

sure. now I get it.

I was reading the readme and thought "oh that is nice. let's see how they manage to execute my ruby code from the go worker" 😄