search

jasmine / jasmine-browser-runner

Serve and run your Jasmine specs in a browser
50 stars 25 forks source link

Running remote grid tests in parallel? #48

Closed aarong closed 8 months ago

aarong commented 8 months ago

I'm trying to run tests on multiple platforms/browsers in parallel on Sauce (useRemoteSeleniumGrid === true). If I call runSpecs() multiple times without allowing it to finish, I understandably get:

Error: listen EADDRINUSE: address already in use :::5555

And if I try to specify a different port for each platform/browser combo, I get:

Error: Can't specify a port when browser.useSauce or browser.useRemoteSeleniumGrid is true

Is there any way to run multiple platforms/browsers in parallel? Apologies if I'm missing something obvious.

sgravrock commented 8 months ago

We could consider removing that restriction. But the experience when you choose a port number other than the handful that Sauce Connect supports is pretty bad:

WebDriverError: Reached error page: about:neterror?e=connectionFailure&u=http%3A//localhost%3A1234/%3Frandom%3Dtrue&c=UTF-8&d=Firefox%20can%E2%80%99t%20establish%20a%20connection%20to%20the%20server%20at%20localhost%3A1234.
    at Object.throwDecodedError (/[...]/node_modules/selenium-webdriver/lib/error.js:524:15)
    at parseHttpResponse (/[...]/node_modules/selenium-webdriver/lib/http.js:601:13)
    at Executor.execute (/[...]/node_modules/selenium-webdriver/lib/http.js:529:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async thenableWebDriverProxy.execute (/[...]/node_modules/selenium-webdriver/lib/webdriver.js:745:17)
    at async Runner.run (/[...]/lib/runner.js:103:5)
    at async Object.runSpecs (/[...]/index.js:112:25)
    at async Command.runSpecs (/[...]/lib/command.js:187:5) {
  remoteStacktrace: 'RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\n' +
    'WebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:191:5\n' +
    'UnknownError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:800:5\n' +
    'checkReadyState@chrome://remote/content/marionette/navigate.sys.mjs:58:24\n' +
    'onNavigation@chrome://remote/content/marionette/navigate.sys.mjs:324:39\n' +
    'emit@resource://gre/modules/EventEmitter.sys.mjs:154:20\n' +
    'receiveMessage@chrome://remote/content/marionette/actors/MarionetteEventsParent.sys.mjs:33:25\n'
aarong commented 8 months ago

I've been trying out various ports by replacing the "5555" in Jasmine Browser Runner's index.js to different values and the options listed on the following page seem to work for me (didn't try them all though):

https://docs.saucelabs.com/secure-connections/sauce-connect/advanced/specifications/

So maybe it would make sense to allow the port to be configured via the options object?

Unless I'm missing a simpler/better way to parallelize my tests across platforms/browsers. Thanks again.