nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.79k stars 1.31k forks source link

Test hangs #1223

Closed naissa12 closed 7 years ago

naissa12 commented 7 years ago

I have a suite of tests. Sometimes, one of the test hangs. It seems like it is not getting a response from selenium. Is there better error handling that can be added when this happens so that the test can retry? Right now it hangs and doesn't retry. It's one test out of the full suite that hangs.

NickStefan commented 7 years ago

Same problem. We need to add a timeout and retry logic to the actual http request between nightwatch and selenium. Setting the page load timeout on seleneium, in a before each, does not work:

browser.timeouts('page load', 30000)

because the "hanging" sometimes happens before even that "before" code runs. This pointed me to go look at the HTTP logic here. There's no timeout or retry logic in the http module of this library.

NickStefan commented 7 years ago

So I think this is actually more related to this: https://github.com/nightwatchjs/nightwatch/issues/470

I monkey patched nightwatch/libs/http/request.js to listen for the timeout event:

self.request.setTimeout(30000, function(){
    console.log('timeout');
     self.emit('error', {});
})

Sure enough, every time our tests had been merely hanging before, this was triggered. Only now, with the error tigger, it helped surface that the selenium server had basically died on the CI server.

I still dont have a solution for selenium server randomly falling over...

NickStefan commented 7 years ago

Now I'm not so sure. Doing health check against the selenium server and it continues to respond to curl requests even when a random test has a timeout

beatfactor commented 7 years ago

Fixed as part of #1232.