Running multiple browsers in parallel

[joeyparrish]

We are trying to replace Karma. With Karma, we can run tests on multiple local or remote browsers in parallel. I would like to make this possible in jasmine-browser-runner. Please let me know what you think of the rough outline below, which I am 100% ready to contribute to if the maintainers are open to this.

Config

As far as the configuration, my first thought is that the "browser" config could be expanded to be an array. Or if the type confusion is too great for that (string, or object, or array of strings or objects...), we could mark "browser" as deprecated and switch to "browsers" (plural) that is always an array of objects.

I would tend to prefer "browsers" with a clear type of array of objects. The only required field on those objects would be "name". So the simplest config:

"browser": "firefox"

Converts to something not much longer:

"browsers": [ {"name": "firefox"} ]

We could continue to provide backward compatibility indefinitely by reading and interpreting "browser" and converting it internally. We could also choose to issue a warning in that case and request that users update their configs, if we ever intend to remove the original field.

The default could remain local Firefox only.

Launching

I propose that the array of browsers be turned into an array of webdriver instances via buildWebdriver, and that this array be passed to RunnerClass in index.js instead of a single instance. Instead of await runTillEnd, the runner could do something like await Promise.all(webdrivers.map(driver => runTillEnd(driver, reporters))).

Reporting

Some work may be needed to clarify which browser is reporting what results.

[sgravrock]

I'm reluctant to build something like this into jasmine-browser-runner. Could you write a wrapper script that forks off N copies of jasmine-browser-runner and specifies the browser via an environment variable like this?

[joeyparrish]

If this is not something you want in jasmine-browser-runner, it might not be the best successor to Karma for us. We will continue the search. Thanks for your time!

[sgravrock]

Thanks for closing the loop. I somewhat expected your response, to be honest: there were always going to be people who need something closer to a feature-parity Karma replacement rather than a lightweight alternative, and it seemed like you might be one of them.

It's too bad that the web-test-runner support for Jasmine that Google teased when they announced the end of Karma development is still (as far as I know) vaporware. I think the combination of that and jasmine-browser-runner would have covered most people's needs.

[joeyparrish]

Losing Jasmine and moving to something like Mocha will be relatively minor. A lot of annoying search-and-replace, but not much serious development.

Losing the ability to test across our entire Selenium grid at once is a bigger deal.