search

calabash / calabash-ios

Calabash for iOS
Other
1.81k stars 369 forks source link

execution expired (HTTPClient::ReceiveTimeoutError) and Host is down (Errno::EHOSTDOWN) random errors #1272

Closed TitouanVanBelle closed 7 years ago

TitouanVanBelle commented 7 years ago

After upgrading my tools to run our automated tests on iOS 10, I am facing errors that seem to happen randomly. Everything works find when using the calabash-ios console.

Environment

OS: Sierra Ruby: 2.2.2 calabash-ios: 0.20.3 run_loop: 2.3.0 Device: iPhone 6s on iOS 10.2.1

$ xcode-select --print-path
/Applications/Xcode.app/Contents/Developer

$ xcodebuild -version
Xcode 8.2.1
Build version 8C1002

$ bundle exec calabash-ios version
0.20.3

Command

This is the command that I run

SCREENSHOT_PATH=build_reports/calabash/screenshots/ LOG_PATH=build_reports/calabash/logs/ RESET_BETWEEN_SCENARIOS=0 WANTS_TO_QUIT=0 DEBUG=1 DEVICE_TARGET='42d83b5db88b4a99b164f9278772779be230f776' DEVICE_ENDPOINT=http://10.90.7.41:37265 HOST_IP=10.90.28.124 BUNDLE_ID=com.fyber.FyberTestApp bundle exec cucumber --tags ~@android-only --tags ~@ignore --tags ~@storekit --tags @actions,@appstart,@banner,@credentials,@installs,@rewarded_video,@interstitial,@offer_wall,@precaching,@sdk,@test_suite,@user_segmentation -r FyberSDK/FyberTestApp/features/ --format pretty --format html --out build_reports/calabash/FunctionalTests.html FyberSDK/FyberTestApp/features/scenarios/

Results

Two Jenkins jobs ran one after the other can have different results even without any changes on the infrastructure or the scenarios in-between.

Run 1 130 scenarios (130 passed) 524 steps (524 passed) Finished in 116m25.628s seconds

Run 2 130 scenarios (15 failed, 115 passed) 524 steps (7 failed, 43 skipped, 474 passed) Finished in 151m39.875s seconds

The second run features this kind of errors

Scenario: Fyber exchange videos are shown   # FyberSDK/FyberTestApp/features/scenarios/publishers/interstitial/video/show.feature:28
 Host is down - Host is down - connect(2) for "10.90.7.41" port 27753 (10.90.7.41:27753) (Errno::EHOSTDOWN)
 ./FyberSDK/FyberTestApp/features/support/launch.rb:47:in `Before'
 ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `call'
 ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `block in <top (required)>'
    Given an Interstitial is shown            # FyberSDK/FyberTestApp/features/step_definitions/product_steps/interstitials_steps.rb:12
    Then I should see the Video Player        # FyberSDK/FyberTestApp/features/step_definitions/product_steps/rewarded_videos_steps.rb:79
    And the video should be played to the end # FyberSDK/FyberTestApp/features/step_definitions/product_steps/interstitials_steps.rb:182
      Host is down - Host is down - connect(2) for "10.90.7.41" port 37265 (10.90.7.41:37265) (Errno::EHOSTDOWN)
      ./FyberSDK/FyberTestApp/features/support/launch.rb:62:in `After'
      ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `call'
      ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `block in <top (required)>'

DEBUG: HTTP: get http://10.90.7.41:27753/1.0/health {:retries=>1, :timeout=>0.5}
DEBUG: HTTP: get http://10.90.7.41:27753/1.0/health {:retries=>1, :timeout=>0.5}
DEBUG: Waiting for DeviceAgent to launch...
SHELL: /Users/qa/.rvm/gems/ruby-2.2.2/gems/run_loop-2.3.0/lib/run_loop/device_agent/bin/iOSDeviceManager install --device-id 42d83b5db88b4a99b164f9278772779be230f776 -a /Users/qa/.rvm/gems/ruby-2.2.2/gems/run_loop-2.3.0/lib/run_loop/device_agent/ipa/DeviceAgent-Runner.app
DEBUG: Took 19.006102 seconds to install DeviceAgent
SHELL: /Users/qa/.rvm/gems/ruby-2.2.2/gems/run_loop-2.3.0/lib/run_loop/device_agent/bin/iOSDeviceManager start_test --device-id 42d83b5db88b4a99b164f9278772779be230f776 >& /Users/qa/.run-loop/xcuitest/ios-device-manager.log
DEBUG: HTTP: get http://10.90.7.41:27753/1.0/health {:retries=>1800, :timeout=>180.0}
  @CMOB-T1419 @smoke
  Scenario: Rotating the device doesn't prevent the video to be played in landscape mode # FyberSDK/FyberTestApp/features/scenarios/publishers/interstitial/video/show.feature:34
 execution expired (HTTPClient::ConnectTimeoutError)
 ./FyberSDK/FyberTestApp/features/support/launch.rb:47:in `Before'
 ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `call'
 ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `block in <top (required)>'
    Given an Interstitial is shown                                                       # FyberSDK/FyberTestApp/features/step_definitions/product_steps/interstitials_steps.rb:12
    When I rotate the device to portrait                                                 # FyberSDK/FyberTestApp/features/step_definitions/device_steps.rb:27
    Then I should see the Video Player                                                   # FyberSDK/FyberTestApp/features/step_definitions/product_steps/rewarded_videos_steps.rb:79
    And the video should be shown in landscape                                           # FyberSDK/FyberTestApp/features/step_definitions/product_steps/rewarded_videos_steps.rb:110
    And the video should be played to the end                                            # FyberSDK/FyberTestApp/features/step_definitions/product_steps/interstitials_steps.rb:182
      Host is down - Host is down - connect(2) for "10.90.7.41" port 37265 (10.90.7.41:37265) (Errno::EHOSTDOWN)
      ./FyberSDK/FyberTestApp/features/support/launch.rb:62:in `After'
      ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `call'
      ./FyberSDK/FyberTestApp/features/support/launch.rb:32:in `block in <top (required)>'

If there is anything I can add let me know.

manjunathscp commented 7 years ago

Experiencing the same issue: execution expired (HTTPClient::ReceiveTimeoutError)

Xcode: Version 8.2.1 (8C1002) iOS: Version 10.2

jmoody commented 7 years ago

Please update to run-loop 2.3.1 and Calabash 0.20.4

mrTwelve12 commented 7 years ago

Experiencing the same issue: execution expired (HTTPClient::ConnectTimeoutError) Xcode：Version 8.2.1（8C1002） iOS：Version 10.1.1 run-loop 2.3.1 Calabash 0.20.4

calabash-ios 0.20.4> start_test_server_in_background
DEBUG: HTTP: get http://192.168.2.10:27753/1.0/health {:retries=>1, :timeout=>0.5}
DEBUG: DeviceAgent-Runner is not running
DEBUG: Sending 'TERM' to ios_device_manager process '34671'
DEBUG: Waiting for ios_device_manager with pid '34671' to terminate
DEBUG: Waited for 0.100279 seconds for ios_device_manager with pid '34671' to terminate
DEBUG: HTTP: get http://192.168.2.10:27753/1.0/health {:retries=>1, :timeout=>0.5}
DEBUG: HTTP: get http://192.168.2.10:27753/1.0/health {:retries=>1, :timeout=>0.5}
DEBUG: Waiting for DeviceAgent to launch...
SHELL: /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/device_agent/bin/iOSDeviceManager install --device-id 97fa0e7c22c406c442ba682dcb70c8260a35f9e9 -a /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/device_agent/ipa/DeviceAgent-Runner.app
DEBUG: Took 12.942645 seconds to install DeviceAgent
SHELL: /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/device_agent/bin/iOSDeviceManager start_test --device-id 97fa0e7c22c406c442ba682dcb70c8260a35f9e9 >& /Users/power/.run-loop/xcuitest/ios-device-manager.log
DEBUG: HTTP: get http://192.168.2.10:27753/1.0/health {:retries=>1800, :timeout=>180.0}
HTTPClient::ConnectTimeoutError: execution expired
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient/session.rb:611:in `initialize'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient/session.rb:611:in `new'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient/session.rb:611:in `create_socket'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient/session.rb:755:in `block in connect'
    from /Users/power/.rvm/rubies/ruby-2.1.4/lib/ruby/2.1.0/timeout.rb:91:in `block in timeout'
    from /Users/power/.rvm/rubies/ruby-2.1.4/lib/ruby/2.1.0/timeout.rb:101:in `call'
    from /Users/power/.rvm/rubies/ruby-2.1.4/lib/ruby/2.1.0/timeout.rb:101:in `timeout'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient/session.rb:748:in `connect'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient/session.rb:511:in `query'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient/session.rb:177:in `query'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient.rb:1242:in `do_get_block'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient.rb:1133:in `protect_keep_alive_disconnected'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient.rb:856:in `request'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/httpclient-2.8.3/lib/httpclient.rb:743:in `get'
... 1 levels...
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/http/retriable_client.rb:163:in `block in request'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/http/retriable_client.rb:150:in `times'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/http/retriable_client.rb:150:in `request'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/http/retriable_client.rb:76:in `get'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/device_agent/client.rb:1244:in `health'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/device_agent/client.rb:1303:in `launch_cbx_runner'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/device_agent/client.rb:271:in `launch'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop/device_agent/client.rb:147:in `run'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/run_loop-2.3.1/lib/run_loop.rb:113:in `run'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/calabash-cucumber-0.20.4/lib/calabash-cucumber/launcher.rb:408:in `block in new_run_loop'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/calabash-cucumber-0.20.4/lib/calabash-cucumber/launcher.rb:406:in `times'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/calabash-cucumber-0.20.4/lib/calabash-cucumber/launcher.rb:406:in `new_run_loop'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/calabash-cucumber-0.20.4/lib/calabash-cucumber/launcher.rb:365:in `relaunch'
    from /Users/power/.rvm/gems/ruby-2.1.4/gems/calabash-cucumber-0.20.4/lib/calabash-cucumber/core.rb:1567:in `start_test_server_in_background'
    from (irb):2

Can only start the first run

jmoody commented 7 years ago

Best guess: the physical device and the the host machine (your mac) are not on the same network.

mrTwelve12 commented 7 years ago

Is under the same network.Network timeout is accidental I delete deviceAgent before every run cucumber, if not delete the deviceAgent cucumber always request timeout

jmoody commented 7 years ago

Try updating to ruby 2.3.1.

This problem is happening on a CI machine and involves physical devices.

My policy to not answer questions about CI troubles because it is impossible for me to know what your set up is.

I delete deviceAgent before every run cucumber,

I don't recommend doing this. We believe we have resolved the issues that made this necessary for some users.

jmoody commented 7 years ago

@TitouanVanBelle

Host is down - Host is down - connect(2) for "10.90.7.41" port 37265 (10.90.7.41:37265) (Errno::EHOSTDOWN) ./FyberSDK/FyberTestApp/features/support/launch.rb:62:in `After'

^ investigate what is happening in your After hook.