[KIEKER-1008] Check test coverage and act accordingly

[rju]

JIRA Issue: KIEKER-1008 Check test coverage and act accordingly Original Reporter: Andre van Hoorn

---

Currently, Kieker includes unit tests (JUnit), as well as integration tests (ant targets, user guide examples). Test coverage, especially in terms of JUnit tests, must be enhanced.

First of all, we need an automated way to check and report test coverage (integrated into the build system).

[rju]

author Marius Löwe -- Tue, 13 Sep 2011 05:36:17 +0200

At first a collection of test coverage tools with ant support:

	= Open Source =	= Latest News/Release =	= Special CI integration =
= Cobertura =	Y	12.03.2010	N
= Emma =	Y	22.06.2005	N
= Hansel =	Y	10.04.2006	N
= GroboCoverage =	Y	05.11.2004	N
= Quilt =	Y	20.10.2003	N
= Serenity =	Y	13.05.2010	Hudson/Jenkins
= CLOVER =	N (no cost)	02.06.2011	Bamboo, Hudson
= Kalistick =	N		Hudson, Bamboo, CruiseControl, Continuum, Others
= Java Quality =	N		CruiseControl, Hudson, other

[rju]

author André van Hoorn -- Mon, 19 Sep 2011 17:04:25 +0200

Given the license, release date, and Hudson integration, we should give Serenity a try.

[rju]

author Marius Löwe -- Sat, 24 Sep 2011 12:39:03 +0200

I added Serenity to the run-tests-junit target. Furthermore I created the kieker-src-jar target which creates a JAR file in tmp/ containing the kieker sources because Serenity needs them in a JAR file.

The output will be placed in the folder serenity. This seems to be hard-coded.

To check the result the Serenity plugin must be installed.

[rju]

author Marius Löwe -- Sat, 24 Sep 2011 18:00:59 +0200

Serenity front end does not work with Opera and Chrome.

[rju]

author André van Hoorn -- Mon, 26 Sep 2011 23:30:06 +0200

Integrated Serenity into our Hudson build.

{br}

The results of the last build are accessible via:{br}

http://samoa.informatik.uni-kiel.de/hudson/job/kieker/lastBuild/checkstyleResult

[rju]

author André van Hoorn -- Mon, 26 Sep 2011 23:33:07 +0200

Replying to [mlo|comment:6]:
> Serenity front end does not work with Opera and Chrome.

The output with Firefox (4.0.1) looks strange as well ...

See [/attachment/ticket/109/hudson-serenity-output.png]

[rju]

author André van Hoorn -- Fri, 18 May 2012 10:03:45 +0200

Björn Weißenfels: As discussed on Wednesday; moving ticket assignments from mlo to you

[rju]

author André van Hoorn -- Mon, 3 Sep 2012 15:37:31 +0200

Florian Fittkau: do you know a working tool to check test coverage in a continuous integration environment?

[rju]

author Florian Fittkau -- Tue, 4 Sep 2012 18:13:12 +0200

Untested:
https://wiki.jenkins-ci.org/display/JENKINS/Emma+Plugin

In Eclipse it is named "EclEmma" and worked fine as far as I remember.

[rju]

author André van Hoorn -- Mon, 3 Jun 2013 16:21:38 +0200

see also KIEKER-981 Done as sonar provides test coverage as as metric

[rju]

author André van Hoorn -- Mon, 13 Jan 2014 13:24:29 +0100

• Potential test coverage tools partially evaluated by Nils
• Emma
• EclEmma (45.00% for Kieker)
• JaCoCo
• Clover (Proprietary)

Next step would be to integrate one of these into our CI setup.

[rju]

author nils-christian -- Fri, 28 Mar 2014 16:00:04 +0100

To test this, I integrated Cobertura into Kieker's WebGUI build. It was very easy (thanks to Maven), but I had to exclude one of the tests as it would fail with Cobertura.

Thanks to Arnd, the Jenkins Cobertura plugin visualizes this:
http://build.se.informatik.uni-kiel.de/jenkins/job/kieker-webgui/

[rju]

author rju -- Sun, 18 Feb 2018 20:00:39 +0100

This is also an old ticket. We use Clover /OpenClover in iObserve. We could use that also in Kieker. Setup within gradle is easy.

[rju]

author rju -- Wed, 21 Feb 2018 21:26:32 +0100

I have added clover to the build system. Clover reports can now generated locally with

./gradlew cloverAggregateReports cloverGenerateReport

[rju]

author rju -- Wed, 21 Feb 2018 21:29:05 +0100

There exists an Open Clover Jenkins Plugin as well as an Eclipse Plugin

[rju]

author rju -- Fri, 23 Feb 2018 10:18:26 +0100

Added OpenClover to Jenkins. Works fine. We are considering to add a configuration setting

[rju]

author rju -- Mon, 19 Mar 2018 11:17:46 +0100

The correct call sequence is `./gradlew cloverGenerateReport cloverAggregateReports`-

[rju]

author rju -- Thu, 9 May 2019 09:47:56 +0200

Please add info on the students' work on tests and test coverage. Also send it back, as I also have some tests written by a student using powermock and mockito to reduce unnecessary integration tests.

[rju]

author rju -- Thu, 5 Mar 2020 17:30:42 +0100

We use clover for test coverage.