code coverage

[sbryngelson]

There is surprisingly good support for code coverage with builtin GNU tools. For example, get an HTML output of all the lines hit by a simulation that looks like this

and one can descend into directories and see what files get hit and even what lines get hit.

The way to accomplish this via gfortran is adding the below to the gfortran compile and linker options in CMakeLists.txt

    add_compile_options(
        $<$<COMPILE_LANGUAGE:Fortran>:-fprofile-arcs>
        $<$<COMPILE_LANGUAGE:Fortran>:-ftest-coverage>
    )
    add_link_options(
        $<$<COMPILE_LANGUAGE:Fortran>:-lgcov>
        $<$<COMPILE_LANGUAGE:Fortran>:--coverage>
    )

then execute a simulation as normal. Then descend build/no-debug_no-gpu_mpi-simulation/CMakeFiles/simulation.dir/fypp/simulation, for example.

Then: lcov --gcov-tool gcov-13 --capture --directory . --output-file coverage.info where here gcov-13 is my particular gcov binary (yours may vary, 13 is for GCC13).

Then: genhtml -ignore-errors unmapped --output-directory html coverage.info

This generates stdout and HTML files. For example:

    (use "genhtml --ignore-errors unmapped,unmapped ..." to suppress this warning)
genhtml: WARNING: ('unmapped') no data for line:360, TLA:UNC, file:/Users/spencer/Downloads/MFC-shb/src/simulation/m_fftw.fpp
    (use "genhtml --ignore-errors unmapped,unmapped ..." to suppress this warning)
Processing file simulation/m_global_parameters.fpp
  lines=335 hit=209 functions=4 hit=4
Processing file simulation/m_ibm.fpp
  lines=300 hit=0 functions=10 hit=0
Processing file simulation/m_monopole.fpp
  lines=180 hit=81 functions=4 hit=4
Overall coverage rate:
  lines......: 22.1% (1958 of 8858 lines)
  functions......: 35.4% (81 of 229 functions)

Then: open html/index.html and get the screenshot above.

We can also see lines hit and not hit like this:

The use case here is seeing if the test cases hit all of MFC's lines of code.

Relevant links:

• https://github.com/sebastianbeyer/gcov-fortran-example
• https://github.com/codecov/example-fortran

Example project with GitHub CI: https://github.com/JamesAllsopp/TestFortranCodeCov

And that project's "real" version: https://github.com/AtChem/AtChem2 and https://app.codecov.io/gh/AtChem/AtChem2