Puma plugin :solid_queue throws an error on start

[raphaeljlps]

I'm trying to use puma plugin :solid_queue but I get an error

gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:11:in `block in start': uninitialized constant SolidQueue (NameError)
      SolidQueue::Supervisor.start(mode: :all)
                ^^^^^^^^^^^^

this happens on my MacBook (M1) and in my linux server/container

my solid_queue.yml

production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues:
      - "*"
      threads: 1
      polling_interval: 2
    - queues:
      - real_time
      - background
      threads: 1
      polling_interval: 0.1
      processes: 1

The full stack

Running 'bundle exec puma -t 5:5 -p ${PORT:-3000} -e ${RACK_ENV:-development}'
[62] Puma starting in cluster mode...
[62] * Puma version: 6.4.0 (ruby 3.2.2-p53) ("The Eagle of Durango")
[62] *  Min threads: 5
[62] *  Max threads: 5
[62] *  Environment: deployment
[62] *   Master PID: 62
[62] *      Workers: 4
[62] *     Restarts: (✔) hot (✔) phased
[62] * Listening on http://0.0.0.0:10000/
[62] Use Ctrl-C to stop
/opt/render/project/.gems/ruby/3.2.0/gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:11:in `block in start': uninitialized constant SolidQueue (NameError)
      SolidQueue::Supervisor.start(mode: :all)
                ^^^^^^^^^^^^
    from /opt/render/project/.gems/ruby/3.2.0/gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:9:in `fork'
    from /opt/render/project/.gems/ruby/3.2.0/gems/solid_queue-0.1.1/lib/puma/plugin/solid_queue.rb:9:in `start'
    from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/plugin.rb:24:in `block in fire_starts'
    from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/plugin.rb:22:in `each'
    from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/plugin.rb:22:in `fire_starts'
    from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/launcher.rb:188:in `run'
    from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/lib/puma/cli.rb:75:in `run'
    from /opt/render/project/.gems/ruby/3.2.0/gems/puma-6.4.0/bin/puma:10:in `<top (required)>'
    from /opt/render/project/.gems/bin/puma:27:in `load'
    from /opt/render/project/.gems/bin/puma:27:in `<top (required)>'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:58:in `load'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:58:in `kernel_load'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:23:in `run'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli.rb:492:in `exec'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli.rb:34:in `dispatch'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/cli.rb:28:in `start'
    from /opt/render/project/.gems/gems/bundler-2.4.22/exe/bundle:37:in `block in <top (required)>'
    from /opt/render/project/.gems/gems/bundler-2.4.22/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
    from /opt/render/project/.gems/gems/bundler-2.4.22/exe/bundle:29:in `<top (required)>'
    from /opt/render/project/.gems/bin/bundle:108:in `load'
    from /opt/render/project/.gems/bin/bundle:108:in `<main>'
[62] Detected Solid Queue has gone away, stopping Puma...
[62] - Gracefully shutting down workers...
[70] Early termination of worker
[68] Early termination of worker
[74] Early termination of worker
[72] Early termination of worker
[62] === puma shutdown: 2023-12-20 22:15:18 +0000 ===
[62] - Goodbye!

[djmb]

Hi @raphaeljlps,

I've tested this out on a fresh Rails app and it works fine when I have gem "solid_queue" in the gemfile. If however I do gem "solid_queue", require: false I get the same error as you have. Is that what you have in your setup?

If that's the issue here, adding require "solid_queue" to the plugin won't help as by the time its invoked we are too far on in the Rails initialisation process. You'll need to require the gem from the Gemfile instead.

If that's not what's going on here, do you have anything else you could share about your setup?

[raphaeljlps]

Hey @djmb

In my gem file I have

# background jobs backend
gem 'solid_queue', '~> 0.1.1'

---

I've added require: true to the gemfile and the same behavior continues

If I do rails s in my development machine, it works as expected (meaning, solid queue runs). However, if I run with bundle exec puma -t 5:5 -p ${PORT:-3000} -e ${RACK_ENV:-development} it crashes with the error.

---

# frozen_string_literal: true

source 'https://rubygems.org'

ruby '3.2.2'

# Bundle edge Rails instead: gem "rails", github: "rails/rails", branch: "main"
gem 'rails', '~> 7.1.2'

# The original asset pipeline for Rails [https://github.com/rails/sprockets-rails]
gem 'sprockets-rails'

# Use postgresql as the database for Active Record
gem 'pg', '~> 1.1'

# Use the Puma web server [https://github.com/puma/puma]
gem 'puma', '>= 5.0'

# Use JavaScript with ESM import maps [https://github.com/rails/importmap-rails]
gem 'importmap-rails'

# Hotwire's SPA-like page accelerator [https://turbo.hotwired.dev]
gem 'turbo-rails'

# Hotwire's modest JavaScript framework [https://stimulus.hotwired.dev]
gem 'stimulus-rails'

# Use Tailwind CSS [https://github.com/rails/tailwindcss-rails]
gem 'tailwindcss-rails'

# Build JSON APIs with ease [https://github.com/rails/jbuilder]
gem 'jbuilder'

# Use Redis adapter to run Action Cable in production
gem 'redis', '>= 4.0.1'

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: %i[jruby]

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', require: false

group :development, :test do
  gem 'brakeman'
  gem 'bundler-audit'
  gem 'debug', platforms: %i[mri]
  gem 'rubocop'
  gem 'rubocop-performance'
  gem 'rubocop-rails'
end

group :development do
  gem 'web-console'
  gem 'rack-mini-profiler'

  gem 'derailed_benchmarks'
  gem 'memory_profiler'
  gem 'stackprof'
end

group :test do
  gem 'capybara'
  gem 'selenium-webdriver'
end

# authentication
gem 'rodauth-rails', '~> 1.12'

# background jobs backend
gem 'solid_queue', '~> 0.1.1', require: true

# multitenancy
gem 'acts_as_tenant'

gem 'matrix', '~> 0.4.2'

# apm
gem 'appsignal'

# email provider
gem 'postmark-rails'

[mattpolito]

Also experiencing this. Works as expected in dev but throws the uninitialized constant SolidQueue error in production env.

[djmb]

Thanks @raphaeljlps - I can see that failing as well now.

Loading the application first in config/puma.rb looks like it fixes it:

# config/puma.rb
require File.expand_path("environment", File.dirname(__FILE__))
...

But that's not ideal. We'd maybe will want to shell out to the solid queue rake task or exe (once we add one) instead.

[koenhandekyn]

same issue here : fine with rails s but same exception with below

 bundle exec puma -C config/puma.rb --early-hints -e development

[atstockland]

Same issue here, on Heroku. Why was this closed with no solutions?

[raphaeljlps]

@atstockland there was a solution, we have @djmb workaround and I see a PR by @nashby.

Are you still affected by this? I'm using the latests version on Render without issues and without the workaround.

[atstockland]

Sorry about that--I didn't realize that was the sanctioned work-around. I did try that, but the issue persisted for me. I'll try again, or try after the next update. Thanks so much!

[atstockland]

I must have had a compounding problems going on the first time I tried. I just updated my puma.rb to load the environment and everything works fine now.

Thanks!

[jules-w2]

Hey

the problem is still here (on devlopement & production mode). The gem seems to be loaded but not all these modules. do you have any idea how to load all the modules?

# Procfile
puma:   bundle exec puma -C config/puma/development.rb

# config/puma/development.rb
bind "ssl://0.0.0.0:4000"
workers 2
threads 1, 3
plugin :solid_queue

# solid_queue/lib/puma/plugin/solid_queue.rb

def start(launcher)
  @log_writer = launcher.log_writer
  @puma_pid = $$

  launcher.events.on_booted do
   puts "puma is booted..."
    puts defined?(SolidQueue)             ? "SolidQueue loaded" : "SolidQueue Not loaded"
    puts defined?(SolidQueue::VERSION)    ? "SolidQueue Version: #{SolidQueue::VERSION}" : "SolidQueue: Not loaded"
    puts defined?(SolidQueue::Supervisor) ? "SolidQueue::Supervisor loaded" : "SolidQueue::Supervisor: Not loaded"

    @solid_queue_pid = fork do
      Thread.new { monitor_puma }
      SolidQueue::Supervisor.start(mode: :all)
    end

    in_background do
      monitor_solid_queue
    end
  end

  launcher.events.on_stopped { stop_solid_queue }
  launcher.events.on_restart { stop_solid_queue }
end

01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: Puma starting in cluster mode...
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: * Puma version: 6.4.2 (ruby 3.2.2-p53) ("The Eagle of Durango")
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: *  Min threads: 1
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: *  Max threads: 3
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: *  Environment: development
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: *   Master PID: 97629
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: *      Workers: 2
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: *     Restarts: (✔) hot (✔) phased
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: * Listening on ssl://0.0.0.0:4000
01:45:06 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:06 -0500: Use Ctrl-C to stop
01:45:07 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:07 -0500: - Worker 1 (PID: 97636) booted in 1.13s, phase: 0
01:45:07 puma.1    | [97629] [MacBook-Pro.local] 2024-04-17 01:45:07 -0500: - Worker 0 (PID: 97635) booted in 1.13s, phase: 0
01:45:07 puma.1    | puma is booted...
01:45:07 puma.1    | SolidQueue loaded
01:45:07 puma.1    | SolidQueue Version: 0.3.0
01:45:07 puma.1    | SolidQueue::Supervisor: Not loaded
01:45:07 puma.1    | /Users/test/solid_queue/lib/puma/plugin/solid_queue.rb:18:in `block (2 levels) in start': uninitialized constant SolidQueue::Supervisor (NameError)
01:45:07 puma.1    |
01:45:07 puma.1    |         SolidQueue::Supervisor.start(mode: :all)
01:45:07 puma.1    |                   ^^^^^^^^^^^^
01:45:07 puma.1    |    from /Users/test/solid_queue/lib/puma/plugin/solid_queue.rb:16:in `fork'

[jczyzewski]

In case anyone still has this problem after the #86, here's what helped me: For me, I got the same error as @jules-w2, but only when puma was running in a cluster mode. Error went away once I added preload_app! to puma.rb config (note that this disables phased restarts)

[jules-w2]

In my case i'm using cluster mode and phased restarts for Puma. I was never able to get the plugin to work in development or production modes.

So I wrote a gem to manage Capistrano tasks. It manages SolidQueue as a service in production mode and in development I launch it in Procfile

https://github.com/IMMOSQUARE/immosquare-capistrano?tab=readme-ov-file#solidqueue-integration

[Unit]
Description=SolidQueue for <%= "#{fetch(:application)} (#{fetch(:stage)})" %>
After=syslog.target network.target

[Service]
Type=simple
Environment=RAILS_ENV=<%= fetch(:stage) %>
User=<%= fetch(:solid_queue_user) %>
WorkingDirectory=<%= current_path %>

ExecStart=<%= Capistrano::Immosquare::Helpers.expanded_bundle_command %> exec rake solid_queue:start
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/bin/kill -TERM $MAINPID

StandardOutput=append:<%= shared_path %>/log/solid_queue_access.log
StandardError=append:<%= shared_path %>/log/solid_queue_error.log

RestartSec=1
Restart=on-failure

SyslogIdentifier=<%= fetch(:solid_queue_service_unit_name) %>

[Install]
WantedBy=default.target