dtcenter / MET

Model Evaluation Tools
https://dtcenter.org/community-code/model-evaluation-tools-met
Apache License 2.0
77 stars 24 forks source link

Enhance MET testing infrastructure to compute code coverage statistics #2612

Open JohnHalleyGotway opened 1 year ago

JohnHalleyGotway commented 1 year ago

Describe the New Feature

The Australian BOM requires a certain test coverage percentage in order for software to be run operationally. This requirement applies to both METplus (python requirement of 90+% coverage) and MET (unknown coverage requirements). This task is to setup code coverage computation for MET and consists of multiple steps.

  1. Identify which toolchain to use (consider the GNU gcov utility).
  2. Compile MET with coverage enabled.
  3. Run full set of MET unit tests and write coverage information.
  4. Produce a report to quantify the coverage.
  5. Quantify the impact to runtime performance.
  6. Automate the computation of code coverage via GHA.

Acceptance Testing

List input data types and sources. Describe tests required for new functionality.

Time Estimate

Estimate the amount of work required here. Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the new feature down into sub-issues. Not sure yet.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

Labels

Projects and Milestone

Define Related Issue(s)

Consider the impact to the other METplus components.

New Feature Checklist

See the METplus Workflow for details.

hsoh-u commented 1 year ago

C++ example project for generating test coverage with gcov for SonarQube: https://github.com/sonarsource-cfamily-examples/linux-autotools-gcov-travis-sc