iotaledger / iri-regression

3 stars 5 forks source link

as a developer I would like to get code coverage information #52

Open alon-e opened 5 years ago

alon-e commented 5 years ago

Description

add a tool to the JVMs to get code coverage reports - jacoco can be added to the java command as an agent, and it will write a report. these reports should be later uploaded to a tool like codacy, but this can be later made into another issue :)

Motivation

in order to assess how good these regression tests are, it's important to know which lines of code are hit. and specifically, new code path should always be hit be tests.

Example tutorial: https://carlosbecker.com/posts/production-code-coverage-jacoco/

alon-e commented 5 years ago

this issue can be broken into 4 tasks:

re: aggregation and Codacy - there are 2 solutions:

  1. use API --partial flag to upload a few files. this requires also a final message at the end. the downside of this is that each machine needs the API token, and there still needs to be a process at the end that sends final.
  2. get all files and use jacoco:merge before uploading, the added bonus is that this file can be stored for later.

see "Multiple coverage reports for the same language" https://github.com/codacy/codacy-coverage-reporter#commituuid-detection

*regardless of the chosen solution - the regular unit & integration tests should also be merged.