jenkinsci / llvm-cov-plugin

Jenkins llvm-cov plugin
https://plugins.jenkins.io/llvm-cov/
6 stars 4 forks source link

Coverage data doesn't match report #9

Closed SoftwareApe closed 5 years ago

SoftwareApe commented 5 years ago

Sorry for bothering you again. Now that the API plugin in the dev version is installed I do get a coverage report. However I can't really make sense of these numbers. I can't upload images to GitHub so text will have to do:

I get

LLVM Function

However looking at the totals section in coverage.json it has345 functions and 4 covered (which is expected because I'm only running a single test case).

I could understand if the files outside the repository are filtered, however there are more than the total number of functions which I don't get.

cizezsy commented 5 years ago

@SoftwareApe

This is the structure of llvm json coverage report:

// The json code coverage export follows the following format
// Root: dict => Root Element containing metadata
// -- Data: array => Homogeneous array of one or more export objects
//   -- Export: dict => Json representation of one CoverageMapping
//     -- Files: array => List of objects describing coverage for files
//       -- File: dict => Coverage for a single file
//         -- Segments: array => List of Segments contained in the file
//           -- Segment: dict => Describes a segment of the file with a counter
//         -- Expansions: array => List of expansion records
//           -- Expansion: dict => Object that descibes a single expansion
//             -- CountedRegion: dict => The region to be expanded
//             -- TargetRegions: array => List of Regions in the expansion
//               -- CountedRegion: dict => Single Region in the expansion
//         -- Summary: dict => Object summarizing the coverage for this file
//           -- LineCoverage: dict => Object summarizing line coverage
//           -- FunctionCoverage: dict => Object summarizing function coverage
//           -- RegionCoverage: dict => Object summarizing region coverage
//     -- Functions: array => List of objects describing coverage for functions
//       -- Function: dict => Coverage info for a single function
//         -- Filenames: array => List of filenames that the function relates to
//   -- Summary: dict => Object summarizing the coverage for the entire binary
//     -- LineCoverage: dict => Object summarizing line coverage
//     -- FunctionCoverage: dict => Object summarizing function coverage
//     -- InstantiationCoverage: dict => Object summarizing inst. coverage
//     -- RegionCoverage: dict => Object summarizing region coverage
//
//===----------------------------------------------------------------------===//

This problem is that the number of LLVM Function is calculated by Functions section, but the data in the Summary section sometimes is mismatched with other sections. The line coverage data has the same problem. Line coverage is calculated by Segments section, but it is also mismatched with summary...

Maybe I need to use data in the Summary section first....

cizezsy commented 5 years ago

Sorry, the count of method actually is the count of instantiation, I will fix it. Thanks!