mission_control-jobs fails with new Rails 8.0.0.beta1

[kbachl]

1. Create new rails app with 8.0.0.beta1, add dummy job;
2. Add gem "solid_queue", ">= 1.0.0"
3. Mount under e.g. /jobs -> mount MissionControl::Jobs::Engine, at: "/jobs"
4. Open up /jobs

Error comes:

MissionControl::Jobs::Errors::IncompatibleAdapter in MissionControl::Jobs::QueuesController#index Adapter Async must implement queue_names

private def raise_incompatible_adapter_error_from(method_name) raise MissionControl::Jobs::Errors::IncompatibleAdapter, "Adapter #{ActiveJob.adapter_name(self)} must implement #{method_name}" end end

mission_control-jobs (0.3.2) lib/mission_control/jobs/adapter.rb:154:in raise_incompatible_adapter_error_from' mission_control-jobs (0.3.2) lib/mission_control/jobs/adapter.rb:91:inqueues' mission_control-jobs (0.3.2) lib/active_job/querying.rb:24:in `fetch_queues'

[TopRoupi]

having the same problem

[TopRoupi]

but i am using rails 7.2.1

[Funcke]

In config/envornments/development.rb what's your config.active_job.queue_adapter value? If it's not set it will use :async. If you are setting it manually like this config.active_job.queue_adapter = :solid_queue It should work. Additionally, make sure you have configured the database connection you want to use for your queue. In rails 8.0.0.beta1 it could look something like this:

# database.yml
default: &default
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  primary:
    <<: *default
    database: storage/development.sqlite3
  cache:
    <<: *default
    database: storage/development_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/development_cable.sqlite3
    migrations_paths: db/cable_migrate

And for defining the correct connection you would need to adapt your config/environments/development.rb by adding: config.solid_queue.connects_to = { database: { writing: :queue, reading: :queue } }

[Funcke]

Also, quick edit: in rails 8.0.0.beta1 you have to migrate the solid_queue migrations on the connection you have dedicated it for manually by running bin/rails db:migrate:<connection_name> in the case above it would be bin/rails db:migrate:queue.

[kbachl]

Hi,

I'm a bit puzzled at the moment. I indeed did forget to add config.active_job.queue_adapter = :solid_queue to development.rb.

The database.yml itself looks plain, as it was created by rails 8 beta 1:

default: &default
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  <<: *default
  database: storage/development.sqlite3

My jobs also work and run when I call them. However, even I did a bin/rails db:migrate command, I now get an error saying:

ActiveRecord::StatementInvalid in MissionControl::Jobs::QueuesController#index Could not find table 'solid_queue_jobs'

[kbachl]

PS: after adding config.active_job.queue_adapter = :solid_queue to development.rb my jobs also fail with same error

It seems that rails beta1 somehow makes solid-queue a first citizen but doesnt configure it enough to be ready to use even it seems so partially... need to look deeper into it

[Funcke]

Hi, the second error indicates that the solid queue tables have not been created in the database.

Make sure you have set up a solid queue database for development that uses the data structure provided in db/queue_migrations.rb as mentioned in the example above.

Since the generated development configuration in a new rails 8.0.0.beta1 app uses the :async adapter for solid_queue by default, a corresponding database connection supporting the solid_queue schema has to be provided manually, as well as running the migrations.

[mabras]

My jobs also work and run when I call them. However, even I did a bin/rails db:migrate command, I now get an error saying:

ActiveRecord::StatementInvalid in MissionControl::Jobs::QueuesController#index Could not find table 'solid_queue_jobs'

# config/environments/development.rb
config.solid_queue.connects_to = { database: { writing: :queue } }

[Funcke]

Hi @mabras, What‘s your corresponding database configuration and how did you executw the expected job?

[kbachl]

Ok, so I did a "rails solid_queue:install" and it creates a "queu_schema.rb" that has much in it, e.g.:

ActiveRecord::Schema[7.1].define(version: 1) do
  create_table "solid_queue_blocked_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.string "queue_name", null: false
    t.integer "priority", default: 0, null: false
    t.string "concurrency_key", null: false
    t.datetime "expires_at", null: false
    t.datetime "created_at", null: false
    t.index [ "concurrency_key", "priority", "job_id" ], name: "index_solid_queue_blocked_executions_for_release"
    t.index [ "expires_at", "concurrency_key" ], name: "index_solid_queue_blocked_executions_for_maintenance"
    t.index [ "job_id" ], name: "index_solid_queue_blocked_executions_on_job_id", unique: true
  end

  create_table "solid_queue_claimed_executions", force: :cascade do |t|
    t.bigint "job_id", null: false
    t.bigint "process_id"
    t.datetime "created_at", null: false
    t.index [ "job_id" ], name: "index_solid_queue_claimed_executions_on_job_id", unique: true
    t.index [ "process_id", "job_id" ], name: "index_solid_queue_claimed_executions_on_process_id_and_job_id"
  end

...
  add_foreign_key "solid_queue_ready_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_recurring_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
  add_foreign_key "solid_queue_scheduled_executions", "solid_queue_jobs", column: "job_id", on_delete: :cascade
end

However, when I then run rails db:migrate:queue it gets replaced by this:

# This file is auto-generated from the current state of the database. Instead
# of editing this file, please use the migrations feature of Active Record to
# incrementally modify your database, and then regenerate this schema definition.
#
# This file is the source Rails uses to define your schema when running `bin/rails
# db:schema:load`. When creating a new database, `bin/rails db:schema:load` tends to
# be faster and is potentially less error prone than running all of your
# migrations from scratch. Old migrations may fail to apply correctly if those
# migrations use external dependencies or application code.
#
# It's strongly recommended that you check this file into your version control system.

ActiveRecord::Schema[8.0].define(version: 0) do
end

And nothing in the queue database changes at all. What exactly goes wrong here? Am I supposed to copy that over to a migration file?

[kbachl]

PS: i also tried a

rails db:schema:load:queue --trace

** Invoke db:schema:load:queue (first_time)
** Invoke db:test:purge:queue (first_time)
** Invoke db:load_config (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:load_config
** Invoke db:check_protected_environments (first_time)
** Invoke db:load_config 
** Execute db:check_protected_environments
** Execute db:test:purge:queue
bin/rails aborted!
TypeError: Invalid type for configuration. Expected Symbol, String, or Hash. Got nil (TypeError)

        raise TypeError, "Invalid type for configuration. Expected Symbol, String, or Hash. Got #{config.inspect}"
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/home/user/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-8.0.0.beta1/lib/active_record/database_configurations.rb:183:in `resolve'

[kbachl]

OK! I got it working on rails 8 Beta1!

The step is simply: DONT USE rails solid_queue:install - but instead add the part of the production.rb to your development.rb:

config.active_job.queue_adapter = :solid_queue
  config.solid_queue.connects_to = { database: { writing: :queue } }

Then delete all previous rails queue DBs and config it in your database.yml that way:

development:
  primary:
    <<: *default
    database: storage/development.sqlite3
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate

after this make sure you got the original "queue_schema.rb" file from rails 8!

Then do a rails db:prepare , NOT A rails db:migrate:queue as this will create an empty schema.rb file instead! Then after the db prepare has run you can use the db:migrate as it then will recreate it from the datase itself.

Quite an annoying path to take to have this working in dev...

[rosa]

@kbachl, this behaviour is caused by a bug in Rails that has already been fixed: https://github.com/rails/rails/issues/52829. In general you don't need to go through all that trouble to get Solid Queue working in dev.

As for the error with the async adapter, yes, that's pending: https://github.com/rails/mission_control-jobs/issues/32

[kbachl]

@rosa thanks for the info! I cant believe I hit that bug...

[igrigorik]

Hit the same issue. @kbachl's fix worked — ty.

@rosa what's the best to unwind and reset? Would prefer not to carry this forward in existing project.

[rameerez]

I got the same error running Rails 8.0.0.rc1 – @kbachl's fix worked, thanks a lot!

I agree this is quite cumbersome and counterintuitive to set up to have it working on dev, I'd expect Rails to come with good defaults so that solid_queue would just work out of the box in dev.

[kbachl]

@rosa Sorry to bother you again, but even after I upgraded to Rails 8.0.0.rc1 I had the bug that I descriped above, meaning doing a db:migrate killed somehow my solid queue database?

[rosa]

what's the best to unwind and reset? Would prefer not to carry this forward in existing project.

@kbachl, @rameerez, @igrigorik: have you try running bin/rails db:prepare rather than bin/rails db:migrate:queue, after running bin/rails solid_queue:install and configuring Solid Queue in development (like here)? That should work just fine, and it follows Solid Queue's installation instructions.

but even after I upgraded to Rails 8.0.0.rc1 I had the bug that I descriped above, meaning doing a db:migrate killed somehow my solid queue database?

It seems that's a regression or a different version of https://github.com/rails/rails/issues/52829. But it sounds like a bug in Rails in any case.

[rosa]

Going to close this one as we have https://github.com/rails/mission_control-jobs/issues/32 for the async adapter issue.