search

grosser / parallel_tests

Ruby: 2 CPUs = 2x Testing Speed for RSpec, Test::Unit and Cucumber
3.38k stars 494 forks source link

Gem attempts to use 3rd database even though only 2 were created. #698

Open tomekn opened 5 years ago

tomekn commented 5 years ago

Hi,

I'm running into the following problem after running bundle exec rake parallel:create parallel:prepare parallel:spec

2019-05-08 11:11:16
api
Created database 'app_test2'
2019-05-08 11:11:16
api
Created database 'app_test'
2019-05-08 11:11:17
db
FATAL:  database "app_test3" does not exist
2019-05-08 11:11:17
api
rake aborted!
2019-05-08 11:11:17
api
ActiveRecord::NoDatabaseError: FATAL:  database "app_test3" does not exist
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:688:in `rescue in connect'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:683:in `connect'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:215:in `initialize'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `new'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `postgresql_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:809:in `new_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:853:in `checkout_new_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:832:in `try_to_checkout_new_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:793:in `acquire_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:521:in `checkout'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:380:in `connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:1008:in `retrieve_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:90:in `connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/railties/databases.rake:177:in `block (2 levels) in <top (required)>'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:62:in `block in check_for_pending_migrations'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:60:in `each'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:60:in `check_for_pending_migrations'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:107:in `block (2 levels) in <top (required)>'
2019-05-08 11:11:17
api
/bundle/gems/rake-12.3.2/exe/rake:27:in `<top (required)>'
2019-05-08 11:11:17
api

2019-05-08 11:11:17
api
Caused by:
2019-05-08 11:11:17
api
PG::ConnectionBad: FATAL:  database "app_test3" does not exist
2019-05-08 11:11:17
api
/bundle/gems/pg-1.0.0/lib/pg.rb:56:in `initialize'
2019-05-08 11:11:17
api
/bundle/gems/pg-1.0.0/lib/pg.rb:56:in `new'
2019-05-08 11:11:17
api
/bundle/gems/pg-1.0.0/lib/pg.rb:56:in `connect'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:684:in `connect'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:215:in `initialize'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `new'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `postgresql_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:809:in `new_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:853:in `checkout_new_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:832:in `try_to_checkout_new_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:793:in `acquire_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:521:in `checkout'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:380:in `connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:1008:in `retrieve_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:90:in `connection'
2019-05-08 11:11:17
api
/bundle/gems/activerecord-5.2.0/lib/active_record/railties/databases.rake:177:in `block (2 levels) in <top (required)>'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:62:in `block in check_for_pending_migrations'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:60:in `each'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:60:in `check_for_pending_migrations'
2019-05-08 11:11:17
api
/bundle/gems/parallel_tests-2.28.0/lib/parallel_tests/tasks.rb:107:in `block (2 levels) in <top (required)>'
2019-05-08 11:11:17
api
/bundle/gems/rake-12.3.2/exe/rake:27:in `<top (required)>'
2019-05-08 11:11:17
api
Tasks: TOP => db:abort_if_pending_migrations
2019-05-08 11:11:17
api
(See full trace by running task with --trace)
2019-05-08 11:11:17
db
LOG:  received smart shutdown request
2019-05-08 11:11:17
db
LOG:  autovacuum launcher shutting down
2019-05-08 11:11:17
db
LOG:  shutting down
2019-05-08 11:11:17
db
LOG:  database system is shut down

Have you ever seen this kind of behaviour before? It looks like the counter is going above the number of processes for some reason.

tomekn commented 5 years ago

NB: I have tried to limit the number of processes using the ENV var, and passing in the number of processes to the rakes to no effect.

grosser commented 5 years ago

try

grosser commented 5 years ago

... if that does not help, bundle open parallel_tests and start adding puts to see how it works internally