search

mbj / mutant

Automated code reviews via mutation testing - semantic code coverage.
Other
1.94k stars 152 forks source link

Add timestamp and mutant version to output #369

Open tjchambers opened 9 years ago

tjchambers commented 9 years ago

I am running mutant under rake for a series of modules (500+) and accumulating the logs in the background for subsequent review and analysis.

I am finding it would be beneficial to have the version number of mutant used and a timestamp in the final output for reference purposes should I need to ask for guidance or identify the mutant run against which source code was in place at the time of the run.

mbj commented 9 years ago

@tjchambers As you are talking about anaylsis of these reports, this is painful with the existing test reports. I'll open another enhancement ticket to finally opensource the implementation of a machine readable (and mergable) output.

tjchambers commented 9 years ago

It may be painful to a degree, but the value is worth the pain at this point. Thanks for being cognizant of it. It would be nice to have an accessible status board similar to CodeClimate's test coverage that I could use to focus on low mutation coverage. This is probably OT, but one thing that I alluded to elsewhere would be knowing where a public method is missing a targeted example group. Those are clear cases for me to get significant wins. But that is just my process, not necessarily best practice or what others would desire.

mbj commented 9 years ago

It would be nice to have an accessible status board similar to CodeClimate's test coverage that I could use to focus on low mutation coverage.

I've had such a (commercial) service in development for fun. But I my spare time for that "for fun" projects collapsed to 0 the last 6month. I actually think such a service would NEVER generate any income that is higher than the amount of energy I have to put in, due the very low demand. I think adding machine readable mutation testing artifacts is the best we can do for now.

tjchambers commented 9 years ago

Once things are machine readable, then putting a straightforward UI on it seems like a smaller step. Again - appreciate your recognition. I still rank adding valuable mutations over adding reporting. But that's just me.

mbj commented 9 years ago

Once things are machine readable, then putting a straightforward UI on it seems like a smaller step.

Depends. I wanted it to be more, referencing the (enhanced) mutant meta for explanations on the alive mutations. Adding kill hints, shared with team information about incremental mutation testing etc.

I consider to opensource the prototype I once build. It will not be too hard to adapt it to recent mutant releases.