search

ruby-shoryuken / shoryuken

A super efficient Amazon SQS thread based message processor for Ruby. This project is in MAINTENANCE MODE. Reach me out on Slack link on the description if you want to become a new maintainer.
Other
2.06k stars 280 forks source link

Map messages to a worker #712

Closed coding-red-panda closed 2 years ago

coding-red-panda commented 2 years ago

Hello,

our problem is something similar to https://github.com/ruby-shoryuken/shoryuken/issues/688 We are currently experimenting with the AWS EventBridge that pushes messages on the SQS Queue, and we would like a single worker to actually pick up those messages and process them.

However since we do not control the events/messages, we ofcourse do not have the shoryuken_class field set on these messages, as it's not ActiveJob that's creating or pushing these messages.

Does Shoryken have a possibility to somehow tell it, accept this message, regardless of where it comes from?

phstc commented 2 years ago

You could try a non-ActiveJob worker.

https://github.com/ruby-shoryuken/shoryuken/wiki/Worker-options

These workers can be mapped to a queue regarderless the shoryuken_class value.

coding-red-panda commented 2 years ago

Okay,

we've switched to that one, and that's picking up the messages now. Amazing.

I am running into some small other configuration problems now, and I'm not sure how to explain the problem. I basically set up the following:

However when I try to access the ::Rails.configuration inside the Worker, I receive errors that certain configuration options do not exist on the class, despite them being set in the config/application.rb I've added an initializer to require the worker from lib, so Rails knows about, and Rails should be available to it, because it works in a different app like this...

Will dig into this more, but might open another issue later on if I can't figure this out.