DatabaseCleaner / database_cleaner-active_record

Strategies for cleaning databases using ActiveRecord. Can be used to ensure a clean state for testing.
MIT License
64 stars 63 forks source link

Stale connection in `DatabaseCleaner::ActiveRecord::Truncation#connection` #86

Open dmolesUC opened 1 year ago

dmolesUC commented 1 year ago

Summary

Each instance of the DatabaseCleaner::ActiveRecord::Truncation strategy initializes its @connection field only once, using the ActiveRecord default connection. When the connection is first requested, ActiveRecord checks it out of the connection pool. Various hooks (e.g. ActiveRecord::TestFixtures.teardown_fixtures) can return the connection to the pool, but Truncation is unaware of this and holds onto the connection object.

If another thread checks the same connection out of the pool and calls disconnect!, then when Truncation tries to use the connection to clean the database, it will find that the connection is closed and raise an error.

Steps to reproduce:

  1. In a Rails/PostgreSQL project using RSpec, configure DatabaseCleaner as follows:

    RSpec.configure do |config|
     config.before(:suite) do
       DatabaseCleaner.strategy = :truncation
     end
    
     config.around do |example|
       DatabaseCleaner.cleaning do
         example.run
       end
     end
    end
  2. Write a test that, in a background thread, checks a connection out of the pool, removes it from the pool, and disconnects it, e.g.

    describe 'connection pooling' do
      def do_disconnect
        Thread.new do
          connection_pool = ActiveRecord::Base.connection_pool
          connection = connection_pool.checkout.tap do |conn|
            connection_pool.remove(conn)
          end
          begin
            connection.execute('SELECT 1')
          ensure
            connection.disconnect!
          end
        end
      end
    
      5.times do |i|
        it "test #{i}" do
          ActiveRecord::Base.connection.execute('SELECT 1')
          sleep(0.5)
          do_disconnect if i % 2 == 0
        end
      end
    end

    (This example is obviously quite contrived; I ran into the problem in a more realistic situation. See discussion in https://github.com/bensheldon/good_job/issues/849.)

Expected

Actual

Workaround

Instead of using DatabaseCleaner.cleaning in an around block, explicitly call DatabaseCleaner.clean_with(:truncation) in an after(:each) block:

RSpec.configure do |config|
  config.after(:each) do
    DatabaseCleaner.clean_with(:truncation)
  end
end

Proposed fix

Get a fresh connection in each call to Truncation.clean -- I simulated this with a prepended module and it seems to work:

module Cleaninator
  def clean
    @connection = nil
    super
  end
end

class DatabaseCleaner::ActiveRecord::Truncation
  prepend Cleaninator
end
olehvavryniv commented 1 year ago

@dmolesUC Thanks a lot! Your workaround saved my life! We use DatabaseCleaner in test API for end-to-end testing. Same issue - connection is closed after ~5 mins.