search

SimonKagstrom / kcov

Code coverage tool for compiled programs, Python and Bash which uses debugging information to collect and report data without special compilation options
http://simonkagstrom.github.io/kcov/
GNU General Public License v2.0
709 stars 109 forks source link

Report coverage summary as plain text in the terminal #438

Closed kinow closed 2 months ago

kinow commented 3 months ago

Hi,

Would it be possible to have a summary of the coverage in the terminal?

We are using bats + kcov to run tests with a Makefile. This gets used with a container and/or locally or on an HPC. In some cases we do not have direct access to a browser, or we are working locally running something like make test that runs the Shell test+coverage, and also the Python test+coverage.

For the Python tests, pytest --cov can produce different type of coverage report (including XML if needed, cobertura, etc.), but by default it emits the output to the terminal. I didn't find a way to get the output for kcov, only the JSON, XML, HTML files, so I resorted to installing jq in the container and processing coverage.json to extract the overall coverage and the per-file coverage, e.g.

jq -r '. | \"Overall coverage: \(.percent_covered)%\"' ./coverage/bats/coverage.json && \
jq -r '.files[] | \"\(.file): \(.percent_covered)%\"' ./coverage/bats/coverage.json
SimonKagstrom commented 3 months ago

Yes, should not be too hard, although something like that will be disabled by default (since there really shouldn't be any output apart from the program itself).

However, what should be outputted? Overall coverage and per-file coverage like in your example?

kinow commented 3 months ago

Yes, should not be too hard, although something like that will be disabled by default (since there really shouldn't be any output apart from the program itself).

Sounds excellent! When I couldn't find the coverage in the console output, I checked the help of kcov, searching for maybe a param to enable it. That option would be enough, even if turned off by default.

However, what should be outputted? Overall coverage and per-file coverage like in your example?

For me that'd be good. Whenever I am working on bats tests, I write the test that I expect to cover a part of the script, and then re-run kcov+bats. Only the overall would already be useful for me to confirm something changed since my last run.

But having the list of files, I could quickly do a kcov ... bats ... | grep sources/script.sh or so, to confirm the coverage of the file being tested changed.

But my use case is specific for writing unit tests with bats and using kcov for coverage. We can also wait to see if others will chime in and comment what they prefer.

Thank you!