Net::ReadTimeout (Net::ReadTimeout) when I use browser.goto

[Leandro-b-03]

So I'm porting my cucumber project to lapis-lazuli framework, and every time I enter on product page using the goto function from browser I receive the error Net::ReadTimeout.

When(/^I add a product "([^"]*)" in the cart$/) do |product|
  @browser.goto @url + '/' + product
  @page.(PdpScreen).add_to_cart(product)
end

The solution I found, but it doesn't seem right since the original cucumber project doesn't throw the Net::ReadTimeout exception:

When(/^I add a product "([^"]*)" in the cart$/) do |product|
  begin
    @browser.goto @url + '/' + product
  rescue Net::ReadTimeout => e
    @browser.goto @url + '/' + product
  end

  @page.(PdpScreen).add_to_cart(product)
end

So anyone with a similar issue or knows why this happens?

[mark0203]

Net::ReadTimeout is a very generic message and can be caused by a bunch of things. Can you give a bit more background info?

• Ruby version
• Browser + version
• Webdriver version
• Gems + versions

Because I see this just working fine in irb (ruby 2.3.3p222, chrome v63, webdriver v2.33, lapis_lazuli 2.0.1)

irb(main):001:0> require 'lapis_lazuli'
=> true
irb(main):002:0> include LapisLazuli
=> Object
irb(main):003:0> browser :chrome
DevTools listening on ws://127.0.0.1:12001/devtools/browser/0e22bb9c-6c09-4c06-a7f1-1ecf8dc67271
=> #<LapisLazuli::Browser:0x4dff688 @browser_wanted=:chrome, @optional_data={}, @browser_name="chrome", @browser=#<Watir::Browser:0x6b33468a url="data:," title="">>
irb(main):004:0> browser.goto "google.com"
=> "http://google.com"
irb(main):005:0>

[Leandro-b-03]

@mark0203, hey man, sure, here is the info:

Ruby 2.3.3p222 (2016-11-21 revision 56859) [x64-mingw32] Firefox v57.0.4 + geckodriver-v0.19.1 Google Chrome v63.0.3239.132 + chromedriver-v2.35

Using rake 12.3.0
Using concurrent-ruby 1.0.5
Using i18n 0.9.1
Using minitest 5.11.1
Using thread_safe 0.3.6
Using tzinfo 1.2.4
Using activesupport 5.1.4
Using activemodel 5.1.4
Using arel 8.0.0
Using activerecord 5.1.4
Using mimemagic 0.3.2
Using mini_portile2 2.3.0
Using nokogiri 1.8.1 (x64-mingw32)
Using systemu 2.6.5
Using macaddr 1.7.1
Using uuid 2.3.8
Using allure-ruby-adaptor-api 0.7.0
Using builder 3.2.3
Using backports 3.11.0
Using cucumber-tag_expressions 1.1.1
Using gherkin 5.0.0
Using cucumber-core 3.1.0
Using cucumber-expressions 5.0.7
Using cucumber-wire 0.0.1
Using diff-lcs 1.3
Using multi_json 1.13.0
Using multi_test 0.1.2
Using cucumber 3.1.0
Using allure-cucumber 0.5.8
Using bundler 1.16.1
Using ffi 1.9.18 (x64-mingw32)
Using childprocess 0.8.0
Using cnpj_utils 1.0.1
Using coderay 1.1.2
Using colorize 0.8.1
Using connection_pool 2.2.1
Using cpf_utils 1.2.1
Using rails-env-switcher 0.2.4
Using cucumber-console 0.2.1
Using daemons 1.2.6
Using titulo_eleitor_utils 1.0.0
Using documentos_br 1.0.3
Using unf_ext 0.0.7.4 (x64-mingw32)
Using unf 0.1.4
Using domain_name 0.5.20170404
Using eventmachine 1.2.5 (x64-mingw32)
Using facets 3.1.0
Using faker 1.8.7
Using multipart-post 2.0.0
Using faraday 0.13.1
Using faraday_json 0.1.4
Using faraday_middleware 0.12.2
Using http-cookie 1.0.3
Using jbuilder 2.7.0
Using json 2.1.0
Using multi_xml 0.6.0
Using rubyzip 1.2.1
Using selenium-webdriver 3.8.0
Using require_all 1.5.0
Using tai64 0.0.5
Using teelogger 0.5.1
Using thor 0.20.0
Using watir 6.10.2
Using lapis_lazuli 2.1.1 from https://github.com/spriteCloud/lapis-lazuli (at development@cbfedb8)
Using mime-types-data 3.2016.0521
Using mime-types 3.1
Using netrc 0.11.0
Using rest-client 2.0.2 (x64-mingw32)
Using mailgun-ruby 1.1.8
Using method_source 0.9.0
Using mysql2 0.4.10 (x64-mingw32)
Using pry 0.11.3
Using slop 3.6.0
Using pry-remote 0.1.8
Using psych 3.0.2 (x64-mingw32)
Using rack 2.0.3
Using rack-protection 2.0.0
Using redis 4.0.1
Using rspec-support 3.7.0
Using rspec-core 3.7.1
Using rspec-expectations 3.7.0
Using rspec-mocks 3.7.0
Using rspec 3.7.0
Using seed_dump 3.2.4
Using sequel 5.4.0
Using sidekiq 5.0.5
Using sqlite3 1.3.13 (x64-mingw32)
Using syntax 1.2.2
Using thin 1.7.2
Using tzinfo-data 1.2017.3
Using webdriver-user-agent 7.4
Bundle complete! 27 Gemfile dependencies, 91 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

I'm using lapis_lazuli 2.1.1 because cucumber 3

Also here my .env

################################################################################
# Copyright 2017 spriteCloud B.V. All rights reserved.
# Generated by LapisLazuli, version 2.0.1
# Author: "Leandro Bezerra" <leandro.bezerra>
###################################################################################
require 'rubygems'
require 'selenium-webdriver'
require 'logger'
require 'uri'
require 'time'
require 'open3'
require 'watir'
require 'mysql2'
require 'webdriver-user-agent'
require 'lapis_lazuli'
require 'lapis_lazuli/cucumber'
require 'active_record'
require 'allure-cucumber'

###################################################################################
# Start the logger
$log = Logger.new('logs/selenium.log')
# Selenium::WebDriver.logger.level = :info

###################################################################################
# Read the config file (load config_local.yml if it exists)
begin
  CONFIGS = YAML.load_file("config/config_local.yml")
rescue
  CONFIGS = YAML.load_file("config/config.yml")
end

###################################################################################
# Load the global variables
$T_START = Time.now

###################################################################################
# Start DB to load some configs
config = YAML::load_file("config/database.yml")["development"]

client = ActiveRecord::Base.establish_connection(config)

###################################################################################
# Config reports
if ENV['LOGREPORT']
  include AllureCucumber::DSL
end

AllureCucumber.configure do |c|
  c.output_dir = "allure"
end

###################################################################################
# Print gem information
print "# --------------------------- VERSION INFO ----------------------------\n"
print "# Lapis Lazuli: #{Gem.loaded_specs['lapis_lazuli'].version}\n"
print "# Selenium Webdriver: #{Gem.loaded_specs['selenium-webdriver'].version}\n"
print "# Watir: #{Gem.loaded_specs['watir'].version}\n"
print "# --------------------------- VERSION INFO ----------------------------\n\n"

###################################################################################
# Launch the browser. Can be firefox, chrome, safari or ie. Defaults to firefox
# Define what browser to use from cucumber yml or from the command line using BROWSER=<..>.

Watir.default_timeout = 30 # Change the default Timeout, default: 60

if Gem.win_platform? # If windows
  ENV['HTTP_PROXY'] = ENV['http_proxy'] = nil

  # Firefox
  firefoxdriver_path = 'browsers/geckodriver.exe'
  raise "Cannot found FirefoxDriver on #{firefoxdriver_path}" if !File.file?(firefoxdriver_path)
  Selenium::WebDriver::Firefox.driver_path = firefoxdriver_path

  # Chrome
  chromedriver_path = 'browsers/chromedriver.exe'
  raise "Cannot found ChromeDriver on #{chromedriver_path}" if !File.file?(chromedriver_path)
  Selenium::WebDriver::Chrome.driver_path = chromedriver_path
end

LapisLazuli::WorldModule::Config.add_config("config/config.yml")
World(LapisLazuli)

# Do something when LapisLazuli is started (This is before the browser is opened)
LapisLazuli.Start do
  #If BROWSER is NIL, Lapis Lazuli will default to Firefox
  if (!ENV['BROWSER'] || ENV['BROWSER'] == 'firefox')
    # Get Selenium to create a profile object
    require 'selenium-webdriver'
    profile = Selenium::WebDriver::Firefox::Profile.new

    # These settings prevent a warning after authenticating via URL
    # For example user:pass@https://website.com/
    profile['network.http.phishy-userpass-length'] = 255
    profile['network.http.use-cache'] = false
    profile.assume_untrusted_certificate_issuer = false
    profile.secure_ssl = false 

    if (ENV['HEADLESS'] == 'true')
      # Start the browser with these settings
      browser ENV['BROWSER'], headless: true, :profile => profile
    else
      browser ENV['BROWSER'], :profile => profile
    end
  end

  if (ENV['BROWSER'] == 'chrome')
    if (ENV['HEADLESS'] == 'true')
      browser ENV['BROWSER'], headless: true, :switches => %w[--ignore-certificate-errors --disable-popup-blocking --disable-translate --disable-notifications]
    else
      browser ENV['BROWSER'], :switches => %w[--ignore-certificate-errors --disable-popup-blocking --disable-translate --disable-notifications]
    end
  end

  if (ENV['FULLSCREEN'] == "true")
    browser.driver.manage.window.maximize
  end
end

# Returns the error as a string if one is detected
# Here you can also insert custom checks. E.g detect if there is any error box and pass the content
def error_on_page?
  begin
    page_text = browser.html
    CONFIGS['error_strings'].each do |error|
      if page_text.scan(error)[0]
        return page_text.scan(error)[0]
      end
    end
  rescue
    $log.debug "Cannot read html for page #{browser.url}"
  end
  return false
end

# Actions that will happen before every scenario
Before do
  # things that should be run before every run
  @browser = browser
  @page = lambda {|b, klass| klass.new b }.curry.(browser)

  @url = ENV["URL"]

  @step_pause_time = CONFIGS['step_pause_time']
  @fullscreen = ENV['FULLSCREEN']
end

# This is executed after every scenario.
After do
  # browser.take_screenshot

  # Check if one of the error strings from the config file is detected on the page
  errors_on_page = error_on_page?
  if errors_on_page
    error "'#{errors_on_page}' found on <a href='#{browser.url}' target='_blank'>page</a>"
  end

  # Wait the required time
  sleep CONFIGS['step_pause_time'] rescue sleep 0
  # browser.cookies.clear
  # browser.close
end

at_exit do
  # Get the actual run on db
  run = Run.where(:name => ENV['REPORT']).first

  if (!run.finished)
    run.running = 0
    run.finished = 1
    run.save
  end

  client_used = Client.where(:run_name => ENV['REPORT'])

  client_used.each do |client|
    client.destroy
  end

  file = 'allure/' + ENV['REPORT'] + '/environment.properties'

  File.open(file, "a+") do |f|
    f.puts('test.run.id=' + ENV['UUID'])
    f.puts('test.run.name=' + ENV['REPORT'])
    f.puts('test.run.url=' + ENV['URL'])
    f.puts('my.properties.browser=' + ENV['BROWSER'])
    f.puts('my.properties.headless=' + ENV['HEADLESS'])
    f.puts('my.properties.fullscreen=' + ENV['FULLSCREEN'])
    f.puts('my.first.url=http://celery.centauro.dc.sbnet')
    f.puts('my.second.url=http://celery.centauro.dc.sbnet')
  end

  @report = ENV['ALLUREREPORT']

  puts @report
  Open3.popen3(@report) do |stdin, stdout, stderr, wait_thr|
    while line = stdout.gets
      puts line
    end

    stdin.close # we pass everything via command-line, so close child's stdin at once
    still_open = [stdout, stderr]  # Array that only contains the opened streams
  end

  if (ENV['SENDEMAIL'] != 'no')
    group_send = ENV['SENDEMAIL'].split(',')

    emails = []
    group_send.each do |group|
      emails_send = EmailReport.where(:report_group_id => group)

      emails_send.each do |email|
        emails.push(email.email)
      end
    end

    send_email(ENV['REPORT'], emails)
  end

  print "# Finished\n"

  # browser.cookies.clear
  # browser.close
end

[sjieg]

Happy to hear you're already trying out LL2.1.1! Please note that this version is still in development. Could you try the following:

1. Run cmd
2. Go to your projects folder
3. execute lapis_lazuli create <projectname> where <projectname> can be something like testproject
4. cd testproject/
5. bundle exec cucumber

If this works as expected, then you have a basis on how it should work. You might be able to figure out why things are breaking for you. Let us know how things turn out.

Cheers!

[Leandro-b-03]

Hey @sjieg Yup, I'm trying LL 2.1.1 for like 2 days ago, and it happens before it as well on LL 2.0.1. I run the project and here the failed scenarios, but none of them as Net::readTimeout

Failing Scenarios:
cucumber features/1_basic.feature:39 # Scenario: example_04 - confirming there is a no results page
cucumber features/2_account.feature:11 # Scenario: account_01 - Logging in
cucumber features/2_account.feature:17 # Scenario: account_02 - Logging out
cucumber features/2_account.feature:23 # Scenario: account_03 - Opening the registration form
cucumber features/2_account.feature:35 # Scenario: account_05 - Logging in a new registration
cucumber features/3_todo_list.feature:7 # Scenario: todo_01 - adding a todo item

I see that the project create a helper folder, it from LL framework? I have some similar, Screen folder, would be more reliable to trade it for Helpers?

Also, now I'm thinking that could be the site that I'm trying to automate...

Thanks for the quick answers...

[mark0203]

I receive the error Net::ReadTimeout

Also, now I'm thinking that could be the site that I'm trying to automate...

@Leandro-b-03, I'm just a user not a developper, but I can give you my thoughts based on what happened on my projects with this similar issue.

I've seen some mismatch between webdriver and browser that gave me this same error.

Also I've seen some timing issues. I noticed you have this in env.rb Watir.default_timeout = 30 # Change the default Timeout, default: 60, what happens if you set this back to 60 or even 90? I had a similar error when I was testing something on a mobile device via the grid: https://discuss.appium.io/t/net-readtimeout-net-readtimeout-q-what-am-i-missing-a-add-read-timeout-capability/19220/4 so maybe the read_timeout capability solved your issue?

Let us know if you find the cause. Or ask @sjieg if you have more questions, he's the expert on LL.

[sjieg]

I see that the project create a helper folder, it from LL framework? I have some similar, Screen folder, would be more reliable to trade it for Helpers?

The project created by LL is to give people an example. We recommend to use it like this, but feel free to find your own solution, like "Screen folder".

Please let me know when the issue is resolved for you, so we can close the ticket.

[Leandro-b-03]

Also I've seen some timing issues where the test is quit just a second or couple of seconds before a proper response is given back. I noticed you have this in env.rb 'Watir.default_timeout = 30 # Change the default Timeout, default: 60', what happens if you set this back to 60 or even 90?

@mark0203, I try to put 60 seconds, but still happens, I minimize it when I try attempts to get something, with begin, rescue and retry if the number of attempts where equals 3 on the def after goto, but even that way it crashes, somehow I realize the error was on goto and a simple rescue there works, I do not receive the error anymore, which is curious, but it not happen.

I had a similar error when I was testing something on a mobile device via the grid so maybe the https://discuss.appium.io/t/net-readtimeout-net-readtimeout-q-what-am-i-missing-a-add-read-timeout-capability/19220/4 so maybe the read_timeout capability solved your issue?

I've read this thread and I think that is a different problem, to me sometimes I get infinite load page, which does not occur while I navigate at the same page.

The project created by LL is to give people an example. We recommend to use it like this, but feel free to find your own solution, like "Screen folder".

I see, if this gonna be a standard for LL I'll be glad to change my code to it, just to maintain the pattern

Thanks, for the answer both of you @mark0203 and @sjieg

[mark0203]

Over here https://github.com/watir/watir/issues/573 Titus says:

Net::ReadTimeout errors usually mean that the driver is not returning a response from the command sent to it. Are you using the most recent 32 bit version of the driver?

Add Selenium::WebDriver.logger.level = :info before your code and re-run. If there is a command being sent to the window endpoint that isn't returning a value, then this is an driver bug, and needs to be filed in the Selenium project along with the applicable details from the logger.

Maybe the logger gives a clue about the issue?

[Leandro-b-03]

Maybe the logger gives a clue about the issue?

Nothing, no clue...

Thanks for the help @mark0203, really appreciate!

[Leandro-b-03]

@sjieg I'll close the issue, I think may it be an infinite page load problem or even timeout, call goto on rescue resolves the problem, thanks all...

Regards!