rails / solid_queue

Database-backed Active Job backend
MIT License
1.95k stars 130 forks source link

Add explicit development instructions to the README. #390

Open evdevdev opened 1 month ago

evdevdev commented 1 month ago

I'm playing around with a Rails 8 app, and I had a small bit of confusion in getting Solid Queue running in development. Therefore, I wrote out instructions that I think I would have found helpful. Maybe other folks will find these helpful as well?

I know that https://github.com/rails/solid_queue/issues/238 mentioned a coming revamp to how Solid Queue works in development. If this clashes with that, I'd be happy to update it to reflect any other changes.

Juksefantomet commented 2 weeks ago

I was unable to get solid queue to work in development until i saw this PR. the Puma.rb edit is vital to get any output regarding development setup. The base note pointing to a plugin mentions nothing about it being required for a native puma+sqlite setup to even catch solid gems.

rosa commented 2 weeks ago

Thanks @evdevdev, sorry for the delay looking into this. I'm still a bit undecided about Solid Queue in development, which is why I haven't merged or requested changes yet.

@Juksefantomet:

The base note pointing to a plugin mentions nothing about it being required for a native puma+sqlite setup to even catch solid gems.

🤔 What do you mean by "required"? Using the Puma plugin is not mandatory, you can run everything in development without it 😕

Juksefantomet commented 2 weeks ago

@rosa

Prior to adding the plugin entry to puma.rb solid queue config and databases was not created.

Added entries to queue.yml, recurring,yml, database.yml and environment/development.rb did not trigger any results when invoking rails db:prepare or rails db:migrate.

The second i added the plugin entry to puma.rb solid_queue was enabled and database for queue,cable and cache was created for development.

Ubuntu 24.04, rails 8.0.0.rc2, ruby 3.3.3 through rbenv.

Fresh project initialized today as we were prepping a PoC for a new concept at work.

Juksefantomet commented 2 weeks ago

@rosa

I stand corrected.

The culprit for database generation was not the plugin, but the solid queue is not being used until i add the plugin entry.

The reason for databases not being created was database.yml:

failed config:

development:
  <<: *default
  database: storage/development.sqlite3
  cache:
    <<: *default
    database: storage/development_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/development_cable.sqlite3
    migrations_paths: db/cable_migrate

As i simply added the required entries to the existing setup for development i did not add "primary:"

resulting working entry:

development:
  primary:
    <<: *default
    database: storage/development.sqlite3
  cache:
    <<: *default
    database: storage/development_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/development_cable.sqlite3
    migrations_paths: db/cable_migrate

but my rails does not provide any console data regarding solid queue and does not trigger internal scheduling until i add the plugin :solid_queue entry to puma.rb

evdevdev commented 1 week ago

@rosa Thanks for getting back to me.

If it's helpful, my primary motivation for having this run in development is so that I can get familiar with all the ergonomics of debugging, reading the logs, monitoring, and all that good stuff. I find myself feeling cautious or timid around having production behave differently from my local setup. I suppose the big fear is that when something catches fire in prod, I won't be able to exactly reproduce it locally.

Out of curiosity, when you say that you're a bit undecided, what part is up in the air? I really love Solid Queue. Thank you for building it.

rosa commented 1 week ago

Thanks @evdevdev, that's very helpful to know! Thanks so much for your kind words as well, really appreciate them ❤️

Out of curiosity, when you say that you're a bit undecided, what part is up in the air?

Configuring Solid Queue in development out of the box for Rails. It was discarded back when it was added as a default, but I think it'd be a good idea to have it. I need to do some convincing for that 😅

evdevdev commented 1 week ago

For anyone who finds this PR, I just ran into a little bit of a gotcha. The changes I've proposed in this PR will break broadcasting locally. (In order to fix it, you need to also shift ActionCable into the database rather than async.)

@rosa If you have any interest, I'll happily update this PR to include those instructions as well.

rosa commented 1 week ago

Ohhh, @evdevdev, good catch! I think I might be able to figure out this week if Solid Queue configured by default in development is a go or no go, and if it's not, I'll get this PR merged.