Open ncoden opened 6 years ago
For unknown reasons, browserstack-runner sometimes exits with failures while all tests run without errors.
browserstack-runner
For exemple see the following Travis job with mocha: all tests passed (or are skipped) but browserstack-runner exists with failures: 3.
mocha
failures: 3
Using config: /home/travis/build/ncoden/foundation-sites/browserstack.json Launching 18 worker(s) for 18 run(s). [OS X Sierra, Safari 10.1] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 6546ms [Windows 10, Internet Explorer 11.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 14034ms [Windows 8, Chrome 65.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 6260ms [Windows 10, Chrome 66.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 7288ms [Windows 7, Internet Explorer 10.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 15411ms [Windows 10, Internet Explorer 11.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 16303ms [Windows 8.1, Firefox 60.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 10193ms [Windows 7, Opera 53.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 5517ms [ios 7.0, Mobile Safari null] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 18048ms [Windows 10, Edge 17.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 9909ms [Windows 7, Opera 52.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 5855ms [OS X High Sierra, Safari 11.1] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 9277ms [Windows 10, Firefox 61.0 beta] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 10925ms [Windows 10, Edge 16.0] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 11188ms [android 8.0, Android Browser null] Passed: 260 tests, 252 passed, 0 failed, 8 skipped; ran for 7173ms All tests done, failures: 3. Exit with fail due to some tests failure.
I found in the following in trace logs, I don't know why android_browser_4_4 is not recognized, but I don't think this is related.
android_browser_4_4
[android 4.4, Android Browser null] client.createWorker | response: undefined Error: Validation Failed - `device` invalid at IncomingMessage.<anonymous> (/home/travis/build/ncoden/foundation-sites/node_modules/browserstack/lib/client.js:73:9) at IncomingMessage.emit (events.js:187:15) at endReadableNT (_stream_readable.js:1090:12) at process._tickCallback (internal/process/next_tick.js:63:19) Error from BrowserStack: Error: Validation Failed - `device` invalid at IncomingMessage.<anonymous> (/home/travis/build/ncoden/foundation-sites/node_modules/browserstack/lib/client.js:73:9) at IncomingMessage.emit (events.js:187:15) at endReadableNT (_stream_readable.js:1090:12) at process._tickCallback (internal/process/next_tick.js:63:19)
See also:
I checked at the source code. I have no problem with Node, but the code around the worker execution and results processing is very confusing. I am unable to propose a solution for now.
Poke @mohitmun @pulkitsharma07 👋
Bump. Would be good to have a solution for testing Android 4.4.
Bug
For unknown reasons,
browserstack-runner
sometimes exits with failures while all tests run without errors.For exemple see the following Travis job with
mocha
: all tests passed (or are skipped) butbrowserstack-runner
exists withfailures: 3
.I found in the following in trace logs, I don't know why
android_browser_4_4
is not recognized, but I don't think this is related.See also:
Possible solution
I checked at the source code. I have no problem with Node, but the code around the worker execution and results processing is very confusing. I am unable to propose a solution for now.