Consider testing strategy

[trevorturk]

I noticed my inline jobs were (surprise) being run inline during tests. The simplest fix is to move config.active_job.queue_adapter = :async_job into config/environments/production.rb only, which lets Rails fall back to its defaults for dev/test. I wonder if we should document that, at least, but it might be worth considering expanding the project surface something akin to Sidekiq's fake/inline/disable: https://github.com/sidekiq/sidekiq/wiki/Testing.

[ioquatix]

Yes, this makes sense to me.

I'm not sure what the right defaults are, but I think in a testing environment, running jobs (or asserting that they would run) makes sense to me.

However, I'm open to ideas and whatever we come up with would be good to document.

[trevorturk]

After a bit of research, I think I would suggest (at least for now) clarifying in the docs that you should put config.active_job.queue_adapter = :async_job into config/environments/production.rb only, not in config/application.rb. This would assume that you intend to use Rails' built-in test adapters/helpers/etc for testing and development. The upside is that this requires very little work now and it's very simple to understand. The downside is that you won't actually be using/testing async-job except for in production. If you're intending to use/trust the ActiveJob system generally and that async-job is working generally, I think this is basically acceptable.

However, at some point it may be worth researching Sidekiq/Faktory's approach to testing, as they have some config/helpers that may come in handy. I'm a bit turned around, though, because I would have expected ActiveJob adapters to also include support for Rails' testing helpers, but it appears that those require the built in Test adapter. This makes me think Sidekiq/Faktory are doing what's expected (including some custom test helpers) so that's something we could consider down the road. For now, though, I think "just use async-job in production, Rails' defaults in dev/test" is a fine way to proceed.

It gets a bit confusing reading back through all of the existing issues, some new and some old, but here are a few links I followed to help inform my current opinion... perhaps worth a skim to get a feeling for the state of play.

• https://guides.rubyonrails.org/active_job_basics.html
• https://guides.rubyonrails.org/testing.html#testing-jobs
• https://api.rubyonrails.org/v5.2.3/classes/ActiveJob/TestHelper.html
• https://github.com/bensheldon/good_job/pull/955
• https://github.com/rails/rails/issues/37270
• https://github.com/rails/rails/pull/48585
• https://github.com/rails/rails/blob/main/activejob/README.md#supported-queuing-systems
• https://github.com/contribsys/faktory_worker_ruby/pull/17/files
• https://github.com/contribsys/faktory_worker_ruby/blob/main/lib/faktory/testing.rb
• https://github.com/rails/rails/blob/main/activejob/lib/active_job/test_helper.rb#L122

[ioquatix]

Thank you greatly for all these links.

I think we should update the documentation as you suggest.

I'll take a stab at it.

[foca]

Rather than using :async_job only in production, how about still recommending people set it up in config/application.rb, but only override the test environment to use ActiveJob's test adapter?

That way you still get to exercise the library during development.