search

villano-lab / nrCascadeSim

calculating the NR spectrum resulting from neutron-capture cascades.
MIT License
0 stars 1 forks source link

[JOSS Pre-Review]: Code Coverage #49

Closed villaa closed 2 years ago

villaa commented 2 years ago

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

There's CI on Travis-CI against one version of ROOT now. That's a big improvement. It would be nice if there was a known lower bound version of ROOT tested in terms of API dependence and a latest version of ROOT tested as well. Also at the moment the tests are testing the examples, but there isn't coverage reported for the codebase. This isn't a requirement, but it is nice to have so users can have some first heuristic of confidence in the strength of the tests / API stability.
villaa commented 2 years ago

Here is the lcov tool to get code coverage. The output files can be uploaded to the codecov hosting service to get a coverage badge.

villaa commented 2 years ago

This website will be a good one to host our code coverage information.

villaa commented 2 years ago

Here is an example for c++11 and Travis-CI.

nuclearGoblin commented 2 years ago

Based on the http://ltp.sourceforge.net/coverage/lcov/readme.php docs and what we have done so far, here is a general outline of how I think the coverage works:

  1. Set coverage flags -fprofile-arcs -ftest-coverage in any files you want coverage tested on in the Makefile.
  2. Set the linking flag -lcov or --coverage for any binaries that will include what you want tested.
    • OSX doesn't seem to like -lcov - might need --coverage or even both. Still testing this one.
  3. Make the program. It will now track coverage info.
  4. If it exists, clear any existing data with lcov --directory dirname --zerocounters before running your test case. Otherwise, you may get false positives left over from a previous test case.
  5. Run test case normally. As you go, coverage information will be generated each time a new line of code is reached.
  6. Have lcov compile the coverage information with lcov --directory dirname --capture --output-file filename.info
    • This puts together which lines of code in which files were reached during your test case.
  7. From here, it can be reformatted to html using genhtml filename.info -o outputdirname.
    • Now you can view the coverage information at outputdirname/index.html.
villaa commented 2 years ago

@gerudo7 With PR #69 we have this issue mostly sewn up. I will start a release of v1.3.1. One small thing I will try to address in the release is that the current bash method for updating the code coverage information is deprecated and will disappear on Feb. 1 2022--here is the new method.

villaa commented 2 years ago

Resolved by PR #69