[New Best Practice Guide]: Code Coverage

[riverma]

Checked for duplicates

Yes - I've already checked

Describe the needs

Code coverage is a metric that helps developers understand the percentage of their source code that is "covered" or executed by automated tests. Achieving high code coverage can help in identifying areas of the code that might be prone to bugs or other issues.

We need to document and share the manual steps to calculate code coverage for teams to leverage, but also work to streamline and automate the calculation of code coverage.

[mike-gangl]

The unity-py python github build has code coverage generation and reporting built in by using pytest-coverage and the coveralls platform for reporting. just set it up a bit ago and it was pretty fast, free for open source projects, and has some nice reporting options. example of the reporting dashboard is here: https://coveralls.io/github/unity-sds/unity-py

[riverma]

The unity-py python github build has code coverage generation and reporting built in by using pytest-coverage and the coveralls platform for reporting. just set it up a bit ago and it was pretty fast, free for open source projects, and has some nice reporting options. example of the reporting dashboard is here: https://coveralls.io/github/unity-sds/unity-py

Thanks @mike-gangl! For Python projects, that sounds like a winner.

@jl-0 - which all languages do you hope to generate code coverage statistics for on an on-going basis?

[yunks128]

@riverma This is a draft code-coverage-tools page. I incorporated the suggestion from @mike-gangl for Python code coverage tools (pytest-coverage, coveralls). Please let me know if you have other suggestions for tools/languages @jl-0. Thanks!

[yunks128]

@riverma It looks like it may be a good idea to incorporate pytest-coverage into Python Starter Kit if not yet done. Or would it be more appropriate for a continuous testing starter kit? What do you think?

[riverma]

@riverma It looks like it may be a good idea to incorporate pytest-coverage into Python Starter Kit if not yet done. Or would it be more appropriate for a continuous testing starter kit? What do you think?

@jpl-jengelke - what do you think about @yunks128 recommendation about?

[jpl-jengelke]

I think it would be an excellent addition to the Python Starter Kit. I would add it to the tests directory at the top of the repo. I believe the best approach is to show by example how these things work.

[jl-0]

Sorry, missed this thread. This looks good for Python, I am also hoping we can cover Javascript. One of the soft requirements was to have a tool that could record code coverage during execution in the cases where a defined test suite had not yet been developed. I know this is possible with python and node, but was not sure if there was method that did both

[riverma]

One of the soft requirements was to have a tool that could record code coverage during execution in the cases where a defined test suite had not yet been developed. I know this is possible with python and node, but was not sure if there was method that did both

@jl-0 - could you elaborate on what you meant by "code coverage during execution in the cases where a defined test suite had not yet been developed"? How would we test for test case coverage without test cases?

[jl-0]

@riverma In the past I used the python trace module to measure the code coverage of a documented test procedure. The software team has a procedure but didn't build in specific unit tests. It basically gives a report of how many lines were actually executed in each of the files.

[riverma]

@jl-0 - I'm not quite following - what's the format for the "documented test procedure"? Code coverage tools look at function / method references and follow the execution stack to see the portion of code covered - is what you're talking about a tool to scan a text document for references to code function or method names?