It seems like integration tests using capybara are flaky and fail regularly if not run sequentially.
Currently, I suspect the problem in Github Actions environment and/or the fact, that tests run in parallel. When restarting the failed job one after the other, they complete successfully.
When running the tests with DEBUG_JS_TEST=1, I see the following erorr:
[MinitestRetry] retry 'test_0001_the index page works' count: 1, msg: Selenium::WebDriver::Error::SessionNotCreatedError: session not created: Chrome failed to start: exited normally.
(session not created: DevToolsActivePort file doesn't exist)
(The process started from chrome location /opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
test/integration_test_helper.rb:302:in `login_admin'
test/test_helper.rb:84:in `before_setup'
test/integration_test_helper.rb:331:in `before_setup'
It seems like integration tests using capybara are flaky and fail regularly if not run sequentially.
Currently, I suspect the problem in Github Actions environment and/or the fact, that tests run in parallel. When restarting the failed job one after the other, they complete successfully.
When running the tests with
DEBUG_JS_TEST=1, I see the following erorr:I found the following issue in selenium: https://github.com/SeleniumHQ/selenium/issues/12841 However, the recommended fix (adding
-remote-debugging-pipe) did not help for me.I noticed the problem here: https://github.com/dm-drogeriemarkt/foreman_dlm/pull/67 A quick look into foreman_ansible also showed Actions runs showing the same behaviour: