Prepare for Rails 7.2 support (and disable enqueue_after_transaction_commit)

[smudge]

/domain @effron @samandmoore /platform @effron @samandmoore

This anticipates the release of a new feature in Rails 7.2 introduced here: https://github.com/rails/rails/pull/51426

In Rails 7.2, ActiveJob will default enqueue_after_transaction_commit to true, and we would like to explicitly set this to false (retaining transactional enqueues, since delayed is DB-backed).

This PR goes a step further and also raises a UnsafeEnqueueError if, during enqueue, it encounters any job type that has overridden enqueue_after_transaction_commit to true. Worth noting that this exception may still be raised after transaction commit (based on the way the lifecycle hooks are handled), so it is intended first and foremost as a guard during development & testing.

As part of this change I also:

• Added a new Appraisal for the latest Rails main build on GitHub.
• Re-generated the rubocop todo file. There are changes we can pull out & autocorrect, but we can do them in separate PRs.

[ghiculescu]

Other than the fact it would open a second transaction, is there any downside to setting it to true that you're trying to protect against?

[smudge]

@ghiculescu It's mainly that one downside, which to be fair is a big one 😄. This gem is a foundational building block for ensuring that persistence side effects always run when they should (and don't run when they shouldn't) and, more broadly, for maintaining eventual consistency across systems. It is also consumed by other libraries (like journaled) that rely on these guarantees.

So by default we want to maintain all existing message delivery guarantees, and avoid the possibility that some jobs may backslide on Rails 7.2 and lose the DB's ACID compliance.

[jmileham]

Also, if you've got 40 minutes I'd hype @smudge's talk, (also linked from the README), if you want to know more about the core guarantee that delayed offers.

[ghiculescu]

Thanks @smudge. That makes sense.

I assume if you use a separate database for your jobs, then most of the guarantees you describe go out the window. The readme says this is possible but not explicitly supported, at least that's my interpretation.

(We currently use delayed job but are considering switching to delayed.)

[smudge]

The readme says this is possible but not explicitly supported.

Since Delayed::Job is just an ActiveRecord class, it can be configured in an initializer to connect to a different database.

I assume if you use a separate database for your jobs, then most of the guarantees you describe go out the window.

Correct. There are situations where it may make sense (perhaps your application already connects to two databases), but I would go as far as to say that this is the wrong queuing library if the intention is always to directly enqueue into a secondary database.

That said, when pointed at the app's primary database, delayed (and, fwiw, delayed_job_active_record) can be used to support a co-transactional "outbox" to ensure that jobs make it into an external job queue, and I'd consider that a good use of this library's core guarantees.

[ghiculescu]

The readme says this is possible but not explicitly supported.

Since Delayed::Job is just an ActiveRecord class, it can be configured in an initializer to connect to a different database.

FYI, with delayed_job it wasn't that simple, because you can only configure abstract classes to connect to different databases. We ended up with this. I imagine it'll be similar with this gem.