search

axemclion / grunt-saucelabs

Grunt task for running all your browser tests using Sauce Labs
MIT License
181 stars 98 forks source link

Report more info on Job timeout #190

Open cvrebert opened 9 years ago

cvrebert commented 9 years ago

When a job times out, the current error message isn't too detailed/helpful when trying to figure out which browser or URL (or whatever) is problematic, which makes debugging test failures harder:

=> Stopping Tunnel to Sauce Labs
>> Error: After trying 90 times with a delay of 2000s, this job never reached 'complete' status.
>>     at /home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/src/Job.js:157:21
>> From previous event:
>>     at TestRunner.runTest (/home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/src/TestRunner.js:170:8)
>>     at runJob (/home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/src/utils.js:51:31)
>> From previous event:
>>     at /home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/src/utils.js:55:10
>>     at /home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/src/TestRunner.js:76:18
>>     at Array.map (native)
>>     at null.<anonymous> (/home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/src/TestRunner.js:75:30)
>>     at Array.map (native)
>>     at TestRunner.runTests (/home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/src/TestRunner.js:74:8)
>>     at /home/travis/build/twbs/bootstrap/node_modules/grunt-saucelabs/tasks/saucelabs.js:111:27
>> 
Warning: Task "saucelabs-qunit:all" failed. Use --force to continue.
Jonahss commented 9 years ago

Ok. Perhaps a pull request?

gdborton commented 9 years ago

Seeing a similar issue on several of my projects. I thought it was probably related to a Sauce Labs outage or maybe a temporary timeout issue, but it doesn't seem to be going away. Will be looking into it this week.

Jonahss commented 9 years ago

Ok. Looking forward to your findings.

liliakai commented 9 years ago

Also seeing this error on my build after adding some long-running tests. I configured max-duration in my Gruntfile, which makes everything green on Sauce's end, but grunt-saucelabs still times out.

https://travis-ci.org/WhisperSystems/TextSecure-Browser/builds/73111341

liliakai commented 9 years ago

Fixed my build by configuring statusCheckAttempts to bring grunt-saucelabs' effective timeout (statusCheckAttempts * pollInterval) up par with max-duration. If there must be two configs, it might be helpful to warn when they are out of sync...

Jonahss commented 9 years ago

Good point. Are you up for contributing a pull request @liliakai ?

matthew-dean commented 8 years ago

I'm getting this error all the time. I don't have long-running tests, but there are a suite of tests for multiple browsers of course. Regardless, the timeout error happens if the test suite hasn't yet completed (which is every time). How do I tell Sauce Labs that the tests are still running fine?

bardiharborow commented 8 years ago

Was there a resolution regarding this? We're still hitting this over at twbs/bootstrap.

Jonahss commented 8 years ago

@bardiharborow ah sorry to hear that. Perhaps @jlipps can ping somebody to address this?

michaelkantor commented 7 years ago

I'm seeing this as well. Everything works fine if I test 1-3 browsers, but any more and it times out with the above error message. That would seem to invalidate saucelabs entire reason for existing.

michaelkantor commented 7 years ago

Ah... I see there's clarification in the README...

alexweissman commented 7 years ago

This appears to still be an issue. I had to disable all of the sauce tests for Select2 because we were hitting timeout errors that were breaking our build (see https://travis-ci.org/select2/select2/builds/291796937, for example).