Unable to process jobs outside of ActiveJob

[robinvdvleuten]

We're using Faktory in a setup where Rails handles the API and a couple of backend processes are written in Go. These backend processes are receiving their work through a Faktory instance.

In Rails we are pushing new jobs by utilizing (or misusing?) the ActiveJob classes. This allows us to have custom Faktory configuration per job (like queue name etc). But the Faktory adapter for ActiveJob wraps the name of the job before sending it to Faktory (ActiveJob::QueueAdapters::FaktoryAdapter::JobWrapper) so in Go we cannot register the correct handler for it. So currently we have a middleware "unwrapping" the name of the job;

class Faktory::Middleware::Unwrap
  class Client
    def call(payload, _pool)
      wrapped_job_klass = payload.dig('custom', 'wrapped')
      if wrapped_job_klass
        payload['jobtype'] = wrapped_job_klass
        payload['custom'] = payload['custom'].except('wrapped')
      end
      yield
    end
  end
end

I can imagine that more developers will run into this and it does not feel right with Faktory's strength to be a polyglot job processing framework. Thus it would be great to be able to configure the "wrapping" on Faktory itself instead of depending on custom middleware.

[mperham]

Do not use ActiveJob, it is designed for use within Rails only. Use the Faktory::Job API.

On Wed, Jun 9, 2021 at 08:22 Robin van der Vleuten @.***> wrote:

We're using Faktory in a setup where Rails handles the API and a couple of backend processes are written in Go. These backend processes are receiving their work through a Faktory instance.

In Rails we are pushing new jobs by utilizing (or misusing?) the ActiveJob classes. This allows us to have custom Faktory configuration per job (like queue name etc). But the Faktory adapter for ActiveJob wraps the name of the job before sending it to Faktory ( ActiveJob::QueueAdapters::FaktoryAdapter::JobWrapper) so in Go we cannot register the correct handler for it. So currently we have a middleware "unwrapping" the name of the job;

class Faktory::Middleware::Unwrap class Client def call(payload, _pool) wrapped_job_klass = payload.dig('custom', 'wrapped') if wrapped_job_klass payload['jobtype'] = wrapped_job_klass payload['custom'] = payload['custom'].except('wrapped') end yield end endend

I can imagine that more developers will run into this and it does not feel right with Faktory's strength to be a polyglot job processing framework. Thus it would be great to be able to configure the "wrapping" on Faktory itself instead of depending on custom middleware.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/contribsys/faktory_worker_ruby/issues/57, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAWXZKUPEQ4XRWXVMEG5DTR6BKRANCNFSM46MISULQ .

[robinvdvleuten]

Indeed if we do use the include Faktory::Job and the perform_async method, it works as expected.

[manuelvanrijn]

I've added a reference in the wiki page regarding this scenario.