3.2.3 update causes 'ERROR: No worker found for [queue name]'

[dongennl]

hi,

absolutely fine on 3.2.2 everything working. Scenario: Drupal CMS posting messages to SQS, Rails 5.2 app with shoryuken picks up the messages and processes them. 2 queues defined in yml config file, 5 workers.

since update to 3.2.3 all of a sudden the workers break and as soon as a message enters the queue 'ERROR: No worker found for [queue name]'

Workers are defined with "include Shoryuken::Worker" and "shoryuken_options queue: 'queue_name', auto_delete: true, body_parser: :json"

starting Shoryuken with "bundle exec shoryuken -R -C config/shoryuken.yml"

what gives? how is this breaking all of a sudden on this update. When I downgrade to 3.2.2 it's fine again.

[dongennl]

Ok. so just checking what changed in 3.2.3

noticed this commit: bbfa813939f88e4e64beb178df85d9868bbff15f

when I switch eager loading back on in an initializer it works again as before, no missing workers. ::Rails.application.config.eager_load = true

I started my rails app on 5.2.0.rc2, and still using that. Am I missing something that should not require me to switch eager loading back on??

[phstc]

@tap87 any thoughts on this? It seems @dongennl isn't using ActiveJobs, he's using standard workers include Shoryuken::Worker.

@dongennl where are your workers located? Since it isn't doing eager load, you may need to manually require them -r ./my_workers.

[dongennl]

Thanks @phstc , indeed not using ActiveJobs.

normal service restored by using "bundle exec shoryuken -R -C config/shoryuken.yml -r ./app/workers" (added -r ./app/workers)

is there another recommended place to put my workers so I do not have to -r? Moving them to the app/jobs folder does not fix the issue, still have to -r them. This change does not seem like an improvement to me.

[tahirpoduska]

@dongennl Can you share the app so I could test? Also, you might need to autoload/eagerload the workers in your app? Shoryuken shouldn't be forcing the eager load. Check this thread from Sidekiq. config.eager_load_paths += %W(#{config.root}/app/workers)

[phstc]

@tap87 @dongennl is using a mixed setup, loading Rails, but using standard workers. Before he was taking advantage of the eager loading, and Rails was also loading the standard workers. In theory, for standard workers, you should always -r <workers>, but since it was working before because of the eager loader and -R, I guess no one was using -r <workers> when using -R.

I'm not sure if we can do much. I guess it should have been a major release instead.

[dongennl]

Thanks, don't quite understand the whole Sidekiq discussion;

Can't this autoload be configurable in the .yml file (autoload on / off) with default on? Seems the safer option for newbies. Thought the point with Rails is that 'stuff just works' without all this kind of complexity. Would imagine this -r and where to put stuff needs to be clearly explained in the installation instructions, less experienced coders (like me) are going to struggle needlessly otherwise.

[tahirpoduska]

autoload_paths and eager_load_paths are configurations in your rails app, you cant set them.

[dongennl]

hi, I meant make the autoload feature that was in 3.2.2 and earlier switchable in the yml file, but never mind.

Is it possible to put the -r argument in the yml file?? That would simplify things. (not listed on wiki) thanks

[kjanoudi]

This certainly should have been a major release, as it broke our system in production when upgrading from the seemingly minor version of 3.2.2 to 3.2.3. At the very least, an open issue or Readme change would be beneficial.

[phstc]

I'm not sure if we can do much. I guess it should have been a major release instead.

@kjanoudi I agree, but unfortunately, it is too late. Would you mind opening a PR to update the README or creating a wiki page? It is been a while since we discussed this issue, I'm sure you are more updated with the problem and solution then I'm at the moment.

[kjanoudi]

Done. Thanks!

[phstc]

@kjanoudi Looks good! Thank you 🍻

https://github.com/phstc/shoryuken/wiki/ERROR:-No-worker-found-for-queue