Rails 7.1 hangs in github actions when use_transactional_fixtures = true

[michelson]

Hi, we are running our test suite with Rspec, recently upgraded to rails 7.1.1 and we find that the github action hangs the processing, it seems that it is waiting for a transaction to complete. Not sure why but we have found that disabling use_transactional_fixtures fixes the issue. Not sure if maintainers know something about this?

What Ruby, Rails and RSpec versions are you using?

Ruby version: 3.2.2 Rails version: 7.1.1 RSpec version: rspec-core (3.12.2) rspec-rails (6.0.3)

Can you provide an example app?

This is the failing job that has the use_transactional_fixtures = true https://github.com/chaskiq/chaskiq/actions/runs/6499097137/job/17659848634

repo: https://github.com/chaskiq/chaskiq

thanks

[pirj]

What if you monkey patch the ‘transaction’ method and dump the ‘caller’ on each call (and after ‘super’)? It will be very helpful to understand what is opening the transaction.

Where is this message from?

message type 0x43 arrived from server while idle

Just in case - i see DatabaseCleaner, but it seems unused. Can you remove it?

[StanBright]

Just to chime in. My specs are hanging with 7.1.1 when being run through the command line, too.

Ruby version: 3.2.1
Rails version: 7.1.1
RSpec version:
rspec-core (3.12.2)
rspec-rails (6.0.3)

I've spent more than half a day now trying to isolate the issue ... without much success.

*Edit: the specs are somewhat randomly hanging. About once in a few runs. I have a feeling that it could be something related to the way transactions are handled in either Rspec or Rails. Also, I'm using factory_bot. That could be relevant, too.

[michelson]

Hi there, Btw, I've ended up disabling the transactioinal_fixtures and adding databaseCleaner on suite :each. The workaround works fine. And there is no difference on terms of time taken compared with the rspec transaction

[nbelzer]

I am encountering a similar issue after upgrading to Rails 7.1.1, although the context is related to job executions happening outside of the transactions wrapping the test. Posting some of my findings here, although I think the issue is related to Rails.

class User < ApplicationRecord
  after_create_commit do
    ExampleJob.perform_later(self)
  end
end

The reason our test suite hangs on Rails 7.1.1 is because of an after_commit_create hook that enqueues a job with perform_later. In our model test (where the default active_job.queue_adapter = :async), the job is enqueued through the after_create_commit. However, in some cases (once every ~3-4 runs) the job is executed after the transaction encapsulating the test is rolled back. This causes the record that was passed along (and serialised) to no longer exist, which in turn results in the job failing. When the job fails the test suite hangs.

When disabling the transactional_fixtures the tests do indeed work as intended. This is expected as the record will now be available when the job runs after the test completion. However, I don't want to disable transactional_fixtures.

One solution I found was to set the active_job.queue_adapter = :test in the test environment (config/environments/test.rb). This results in no execution of the job unless wrapped with a perform_enqueued_jobs block (which arguably was what I expected to happen originally).

The other, which can be applied to a specific test or group of test is to set the execution of scheduled jobs to happen immediately (if you are using the :async adapter, which is the default) by setting ActiveJob::Base.queue_adapter.immediate = true in a before block.

I've rolled back to Rails 7.0.8 and found that also there the job would fail with the same frequency, however, here it would not result in the test suite hanging if it did. Since we only updated Rails it appears to be an issue caused by rails.

To verify this I re-wrote the specs that fail in minitest, and indeed the hanging behaviour happens there too every 3-4 runs. I'm not sure if this is the same issue that the people above encountered, but for me the issue does not appear to be directly related to rspec. Still I thought it would be worthwhile posting it here for other people that encounter it.

[pirj]

Thanks for the research, @nbelzer !

job executions happening outside of the transactions wrapping the test

Sounds like a bug caused by some race condition.

queue_adapter = :inline … results in no execution of the job

I would expect them to be executed immediately, too. https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/InlineAdapter.html Can you provide a reproduction script?

Would you submit your findings to Rails? I’m uncertain if it is the same issue, but seems telated.

[nbelzer]

I would expect them to be executed immediately, too. https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/InlineAdapter.html

Sorry for the confusion I meant to write active_job.queue_adapter = :test, I also updated the comment above.

Can you provide a reproduction script?

I've created this repo with instructions on how to replicate using Rails 7.1.1 and minitest (since my issue also happens with minitest, and I wanted to stay close to rails defaults for a reproduction) but so far I've been unable to replicate the tests hanging there.

Would you submit your findings to Rails?

I'll need to do some more digging to find a way to replicate the hanging issue in the repo linked above. When I do I'll submit the issue to the Rails repo.

EDIT: Was not able to replicate the hanging so I ended up posting my findings on a related issue on the rails repo.

[OpakAlex]

@nbelzer thanks a lot!