Propose using js-reporters in Karma runner

[Krinkle]

It would be nice if, for example, a plugin like karma-reporter-mocha could be used directly with qunit-cli.

If we establish a common API between QUnit and Karma for attaching reporters, this should be possible.

Karma already supports this in a way by abstracting the interfaces for testing libraries and reporters. In fact, you can already use karma-runner with karma-qunit and karma-reporter-mocha together to achieve this.

[leobalter]

That's a great a idea. Do we have someone from the JSF already connected w/ Karma? Or maybe someone we should ping to ask for support/help?

[Krinkle]

As of last year, I'm contributor on grunt-karma and karma-qunit.

Their Gitter room at https://gitter.im/karma-runner/karma is quite active for support. But, for this purpose we may want to use the issue tracker instead, which tends to get a quicker/better response for technical proposals.

[Krinkle]

Ref https://github.com/karma-runner/karma/issues/1183.

[johnjbarton]

Note that karma isn't a testing framework itself but rather hosts other frameworks. The karma reporter protocol is very narrow and I would push it to base it simply on the current 'info' API, making the karma part of this story be just a pipe between the client browser and server. On the client side, reporter calls would be converted to named json packets and reversed on the server side. Here is an example from karma-jasmine: https://github.com/karma-runner/karma-jasmine/commit/7b73ce0abec8da370986d10cbb040fe6e5ae1d22#diff-e52a9a67956bd51d044af4058fb939ceR202

The jasmine API is pretty complete and internally at my company we convert all reporters to jasmine API. Seems like a good starting point if the jasmine word is just dropped from the function names.

[Krinkle]

@johnjbarton Thanks. Karma's role would indeed be mainly on the consumer/reporting side.

I think we're seeing the same future, but let me reflect in my own words to see if we're thinking the same thing:

• For frameworks, Karma could perhaps support the names and shapes of the Runner API in its "info" protocol. That way, Karma plugins for testing frameworks such as karma-qunit and karma-mocha could simplify their adapter code to just forwarding these events as-is (example). It might even be possible to make it fully declarative, where one would tell karma.conf.js to load qunit from node_modules and Karma would know in a framework-agnostic way that it can get events from QUnit.on() and transport them to the Karma server. There would no longer exist a karma-qunit plugin, perhaps. But even simplifying them would be a step forward.

• For reporters, Karma could support standard CRI reporters. This would widen the ecosystem of reporters that Karma users can use, and conversely it would allow maintainers of Karma reporters to make their packages usable outside Karma. For example, as a user I could use the same Storybook/TAP/JUnit/TeamCity/summary/etc package with Karma when testing code in a browser, and then plug in the same set of reporters into the Jasmine CLI when testing my code in Node.