search

SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.57k stars 8.17k forks source link

Ruby: sporadic "stream closed in another thread" IOError #8413

Closed mattbrictson closed 4 years ago

mattbrictson commented 4 years ago

🐛 Bug Report

We are using selenium-webdriver to test a Rails 6 application using capybara and rspec. The tests will sporadically fail with the following error:

IOError:
  Failed to open TCP connection to 127.0.0.1:9515 (stream closed in another thread)

Judging by the backtrace and the port number, I am assuming this is selenium-webdriver failing to communicate with chromedriver.

Backtrace:

/usr/local/lib/ruby/2.5.0/net/protocol.rb:233:in `write'
/usr/local/lib/ruby/2.5.0/net/protocol.rb:233:in `write0'
/usr/local/lib/ruby/2.5.0/net/protocol.rb:207:in `block in write'
/usr/local/lib/ruby/2.5.0/net/protocol.rb:224:in `writing'
/usr/local/lib/ruby/2.5.0/net/protocol.rb:206:in `write'
/usr/local/lib/ruby/2.5.0/net/http/generic_request.rb:334:in `write_header'
/usr/local/lib/ruby/2.5.0/net/http/generic_request.rb:186:in `send_request_with_body'
/usr/local/lib/ruby/2.5.0/net/http/generic_request.rb:121:in `exec'
/usr/local/lib/ruby/2.5.0/net/http.rb:1495:in `block in transport_request'
/usr/local/lib/ruby/2.5.0/net/http.rb:1494:in `catch'
/usr/local/lib/ruby/2.5.0/net/http.rb:1494:in `transport_request'
/usr/local/lib/ruby/2.5.0/net/http.rb:1467:in `request'
./vendor/bundle/ruby/2.5.0/gems/webmock-3.8.3/lib/webmock/http_lib_adapters/net_http.rb:97:in `block in request'
./vendor/bundle/ruby/2.5.0/gems/webmock-3.8.3/lib/webmock/http_lib_adapters/net_http.rb:105:in `block in request'
./vendor/bundle/ruby/2.5.0/gems/webmock-3.8.3/lib/webmock/http_lib_adapters/net_http.rb:137:in `start_with_connect_without_finish'
./vendor/bundle/ruby/2.5.0/gems/webmock-3.8.3/lib/webmock/http_lib_adapters/net_http.rb:104:in `request'
./vendor/bundle/ruby/2.5.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/default.rb:129:in `response_for'
./vendor/bundle/ruby/2.5.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/default.rb:82:in `request'
./vendor/bundle/ruby/2.5.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/common.rb:64:in `call'
./vendor/bundle/ruby/2.5.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/bridge.rb:167:in `execute'
./vendor/bundle/ruby/2.5.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/bridge.rb:567:in `execute'
./vendor/bundle/ruby/2.5.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/bridge.rb:558:in `find_elements_by'
./vendor/bundle/ruby/2.5.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/search_context.rb:80:in `find_elements'
./vendor/bundle/ruby/2.5.0/gems/capybara-3.32.1/lib/capybara/selenium/extensions/find.rb:17:in `find_by'

To Reproduce

Unfortunately I have not been able to consistently reproduce this error. It seems to fail roughly once out of every 1,000 test cases. On Circle CI this means about 5-10% of our CI builds (Ubuntu) fail due to this "stream closed in another thread" error. I've also seen the error on my development workstation (macOS), although this is more more rare.

The workaround is to use the rspec-retry gem to automatically retry failures when these sporadic errors occur. The test will always succeed on second or third attempt.

Apologies for the vague bug report. If there is another repo where this issue would be more appropriate (capybara? chromedriver?) or suggestions for troubleshooting steps, please let me know.

I've searched for "stream closed in another thread" in various projects but not found any similar reports.

Environment

OS: Ubuntu and macOS Browser: Chrome Browser version: 83.0.4103.97 Browser Driver version: ChromeDriver 83.0.4103.39 Language Bindings version: Ruby 3.142.7 Selenium Grid version (if applicable): N/A

diemol commented 4 years ago

@mattbrictson hard to say, without a way to reproduce this I would not expect to see the Ruby maintainers working on this. Is there a way you can provide something (code-wise) to reproduce the issue?

titusfortner commented 4 years ago

@mattbrictson Whatever our fix is, it won't get released until 4.0, which won't help you immediately. Things you can do in the meantime:

  1. If it is related to this Ruby bug, then updating to Ruby 2.6 should fix it
  2. You could subclass Selenium::WebDriver::Remote::Http::Default, override #response_for and pass in an instance of the subclass as an http_client parameter when initializing the driver.

As for 4.x release, @twalpole / @p0deje can either of you think of a downside to adding IOError to the list of things we rescue/retry in #request?

mattbrictson commented 4 years ago

Is there a way you can provide something (code-wise) to reproduce the issue?

Unfortunately, no, I haven't figured out a repro for this. The rspec-retry workaround is working fine for me, so even if the underlying error doesn't get fixed this is no longer urgent. If you want to close this issue as "cannot reproduce" that is fine.

If it is related to this Ruby bug, then updating to Ruby 2.6 should fix it

Great callout. I am stuck on Ruby 2.5.5 at the moment but this is good news.

You could subclass Selenium::WebDriver::Remote::Http::Default, override #response_for

Thanks for pointing me to this part of the code. Seems like I could gracefully do a retry here when this particular IOError happens. I will give this a shot.

Thanks all!

twalpole commented 4 years ago

@titusfortner Given the age of the bug you linked, I’d be surprised if the fix didn’t get backported to 2.5.x (if it existed in 2.5 too). Retrying once on IOError is probably fine although I’m reticent to add hacks without understanding exactly what’s causing the issue. Obviously a way to repro would help with that.

titusfortner commented 4 years ago

oddly, github says that commit is only included in these tags:

v2_7_1  v2_7_0 v2_7_0_rc2 v2_7_0_rc1 v2_7_0_preview3 v2_7_0_preview2 v2_7_0_preview1 
v2_6_6 v2_6_5 v2_6_4 v2_6_3 v2_6_2 v2_6_1 v2_6_0 v2_6_0_rc2 v2_6_0_rc1 v2_6_0_preview3 v2_6_0_preview2

Agreed, I would be much more comfortable adding it to the retry if we knew it was a result of this Ruby 2.5 bug.

titusfortner commented 4 years ago

I'm going to close this one. If we can get confirmation that Ruby 2.6 fixes the issue, then we can add a retry for IOError specifically for 2.5 users. If anyone else has a similar issue and can provide a reproducible case, we can re-open / address it.