search

bensu / doo

doo is a library and lein plugin to run cljs.test on different js environments.
Eclipse Public License 1.0
324 stars 63 forks source link

Add support for JUnit test reporting #34

Open danielcompton opened 8 years ago

danielcompton commented 8 years ago

It would be nice to have JUnit test reporting as this is the lingua franca of test result reporting. This can then be used in a number of CI tools for more detailed test reporting, beyond just pass/fail.

Something like https://github.com/karma-runner/karma-junit-reporter could be good for test reporting, although it only supports Karma. Perhaps it would be better to create a reporter that can be used for any testing tool, relying on cljs.test report to get details.

bensu commented 8 years ago

I don't think that doo should make test reporting decisions. cljs.test is written with extensibility in mind and it is trivial to add new reporting through cljs.test/report. We can add/integrate a cljs.test JUnit reporter to doo in a different project and document it in a wiki, allowing everybody to choose their own reporting scheme. What do you think?

danielcompton commented 8 years ago

That sounds pretty sensible. I originally thought it would need to be more tightly integrated with doo, but that may not be the case.

kamituel commented 8 years ago

I need to get jUnit compatible output from a tests run with Doo/Karma as well. Could you explain what do you mean by "We can add/integrate a cljs.test JUnit reporter to doo in a different project"? Is there such a reference project anywhere? How could I do that?

I also saw https://github.com/bensu/doo/issues/43 and I like the third approach there (merging custom Karma conf with the default one provided by Doo).

danielcompton commented 8 years ago

@kamituel I use karma directly and a Junit Karma plugin to achieve this.

kamituel commented 8 years ago

@danielcompton Thanks for the reply! What do you mean "directly" - without Doo?

danielcompton commented 8 years ago

Yep, I'll try to write up a blog post on the method.

avescodes commented 6 years ago

@danielcompton Did you ever end up doing this?

danielcompton commented 6 years ago

Still on the backlog sorry...

MatthewDarling commented 6 years ago

There are new alphas out for clojure.data.xml that have CLJS support. That should prove helpful for writing a CLJS JUnit test reporter.

I was able to get the CLJS version of [clojure.data.xml 0.2.0-alpha3] to read in an example output file from test2junit. I did need to remove newlines in between tags though.

I was planning to try to adapt test-report-junit-xml. With a working CLJS JUnit reporter, a project could pull it in and then doo wouldn't have to be concerned about it.

bensu commented 6 years ago

Hey @MatthewDarling

Thanks! That is exactly what I had in mind earlier in the thread. Overriding the default cljs.test reporters is a matter of importing the right multimethods implementing JUnit reporting (or any other format for that matter). The reporter could live on a different project to be required by users that want to use JUnit (optionally that project could be a folder in this repo).

Please ping us back if you do have a working cljs.test implementing JUnit.

avescodes commented 6 years ago

Good stuff. I might have some bandwidth to look at this at work next week. Any code you can share @MatthewDarling?

MatthewDarling commented 6 years ago

There's not much in it, but I've put up what I had so far over here. Basically just proving to myself that parsing + emitting work.

Starting from here is an EDN-syntax example of what a test reporter should produce. Includes a failing test, a passing test, and a test with errors. It's all Java exception style right now, just because I was cribbing off of the test2junit example. But that seems to be the general structure.

I've been testing the output using a Node.js tool called xunit-viewer. You have to override the default 8080 port though, it will exit silently if that port is already taken.

In terms of directly copying redbager/test-report-junit-xml, I think what I've done so far covers the main code for creating the "messages" about the tests. Collecting the messages will be trickier - their version uses robert.hooke to handle the message collection.

I don't think there's a tool like robert.hooke for CLJS...? But I know the cljs.test methods are a bit different from the clojure.test ones, so maybe there's something helpful there.