Support stryker json output

[hcoles]

The stryker team have put together a language agnostic format for displaying mutation results.

Some changes might be required to the format, but It would be good to support this with a view to eventually replacing the aging existing report output.

https://github.com/stryker-mutator/mutation-testing-elements/issues/18

This ought to be relatively straightforward. One possible issue is that pitest (I think, its been a long time since I looked at the reports) currently streams it's output to disk on a per class basis, overwriting the previously generated file as new results come in.

It looks as if stryker requires a single file for all mutants - pitest would need to track how many mutants there are for each file (so classes, inner classes, lambdas etc) and only append to the json when results of them all have been retrieved. (it possible this has already been implemented).

[nicojs]

Thanks for opening this issue! Your explanation already provides a good information source for anyone interested in helping with this feature.

We will make it possible to host the json output from your build in the stryker dashboard for open source projects, so it might be a good idea to implement it in such a way that it can easily be uploaded there later.

👍

[Wmaarts]

Thanks for the explanation, I'll try to have a go at this. :)

[Wmaarts]

Small question: I found this in the hackers_guide.md: "Third party dependencies must not be introduced into the pitest module as they may conflict with those of the code under test."

Does that mean we can't use a library like Jackson for JSON parsing? Is a 'hand-written' JSON parser necessary? 😬 That's what seems to be happening in the XML parser too...

[hcoles]

Dependencies used to be a huge issue as they all had to share the classpath with the code under test. The situation is now improved and this only applies to the things in the main 'pitest' module.

This functionality doesn't need to be in the main module.

I'd suggest putting it into a plugin to start with. This has a couple of disadvantages (requires a bit of weird boiler plate in the build + is less convenient to consume) but is entirely separate/independent of the main codebase so much easier to work on. Plugins can always be pulled into the main codebase later.

It's fine to use dependencies in a plugin. The only caveat is that they will need to be packaged up into a fat jar. If you look at the pitest-html module it does this for StringTemplate using the shade plugin.

(note the html module is a plugin that just happens to be located in the main source tree + gets some special treatment so that it is automatically available).

[Wmaarts]

Thanks for the advice.

I started out a bit messy and just implemented the stryker report generator into the existing html-report package, because that was the easiest to get it working. ^^ I just spend some time extracting it into its own package so it can be moved elsewhere.

It's far from fully tested, but it looks to be working. Tested it on a simple and small code base and here's an example:

[Wmaarts]

I'd suggest putting it into a plugin to start with.

I'm not quite sure how to do this... do you suggest not putting it in the main source like the old html-reporter?

EDIT: I found the documentation and an old example project. Trying to get that to work right now but no success so far.

[hcoles]

The junit-5 plugin might be a better example as this has been kept upto date

https://github.com/pitest/pitest-junit5-plugin

The html module in the pitest codebase is also in effect a plugin and uses the interfaces you will need.

https://github.com/hcoles/pitest/tree/master/pitest-html-report

[Wmaarts]

Development on this here -> https://github.com/Wmaarts/pitest-stryker-report-plugin. :)