Differential coverage, date binning, various new features and refactoring

[henry2cox]

this PR subsumes PR #86 which was created some time ago. It contains all the same code, adds all the same features, and makes all the same changes. The only substantial change vs the 'diffcov_initial' branch mentioned in #86 is (automated) reformatting of code.

The aim of the new branch - and the aim of this PR - is to make it very easy to merge to master.

Commit comment from merge branch says it all:

(Squashed commit of all changes from the 'diffcov_initial' branch:) New features and enhancements (compared to lcov 1.16):

New features and capabilities fall into 6 major categories:

a) Categorization

 This refers primarily to differentical coverage categorization as
 well as date- and owner-binning.  See https://arxiv.org/abs/2008.07947
 or https://ieeexplore.ieee.org/document/9438597 for a detailed
 description of the concepts and discussion of several possible use models.
 A video presentation of the basic ideas can be found at
 http://doi.org/10.5281/zenodo.4653252

 Differential categorization and binning are orthogonal in the sense
 that you can generate differential report without binning as well
 as 'vanilla' coverage reports with binning.  See the above papers
 and the genhtml man page for details.

 Related options:
    --baseline-file, --diff-file, --annotate-script, --date-bins,
    --new-file-as-baseline, --elide-path-mismtach

b) Error handling

 A generic - but very simple - error handler has been added to the
 lcov tool suite.  The error handler is used to report exceptions,
 and provides a mechanism for the user to ignore the particular
 message if desired.
 See the genhtml/lcov/geninfo man pages for details.

 Note that some errors are unrecoverable - and cannot be suppressed or
 ignored.

 Related options:  --ignore-error, --keep-going

c) Navigation and display:

 Navigation aids such as hyperlinks to the first uncovered region,
 to the next uncovered region, etc. have been implemented.  Similarly,
 new tables, new columns, and new links between tables enable the
 user to identify the author of particular code (covered or not
 covered), as well as the time period when the code was written.

 Collectively, these features help the user to quickly identify the
 cause of code coverage issues, and to then decide what to do.

 An option to generate a 'hierarchical' coverage report (which follows
 the source code directory structure) as well as various other small
 features (tooltip popups, user-specified HTML header, footer, and
 table labels, etc.) are also available.

 See the genhtml man page for some details, as well as the
 'gendiffcov/simple' testcases for some examples.

  Related options:
      --baseline-title, --baseline-date, --current-date, --hierarchical,
      --show-owners, --show-noncode, --show-navigation

d) Data manipulation

 Filters are used to suppress or remove certain coverage artifacts -
 for example, branches generated by the compiler (e.g., for exception
 handling).  These artifacts can overwhelm the user code and obscure
 coverage features that are interesting to the user.

 Other options are used to focus on or to exclude certain sections
 of code, as well as to do regexp replacement of file names.
 (Path munging is useful primarily when the build structure does
 not exactly match the layout in your revision control system; this
 is common in large projects with reusable components.)

 See the lcov/geninfo/genhtml man pages for a detailed description of
 the available filters and manipulation features.

 Related options:
    --include, --exclude, --omit-lines, --substitute, --filter

e) Callbacks/customization

 The user can supply callbacks which are used to:

i) interface with the revision control system
   Sample scripts:
     - Perforce:  see 'p4diff' and 'p4annotate'
     - Git: see 'gitdiff' and 'gitblame'
ii) verify that source code versions are compatible, and 
     - Sample scripts: see 'get_signature', 'getp4version' 
iii) enforce a desired code coverage criteria
      -  Sample script: criteria

See the genhtml/lcov/geninfo man pages for details.

Note that the various sample scripts are found in the source code 'bin' directory, but are installed in the $LCOV_HOME/share/lcov/support-scripts directory of the release.

Related options: --annotate-script, --criteria-script, --version-script

f) Performance

 lcov/genhtml/geninfo have been refactored to parallelize computation
 across multiple cores, if requested.
 In general, this provides speedup that is nearly linear in the number
 of cores.
 There is also an option to throttle parallelism to not exceed peak
 memory consumption contraints, as well as options to enable simple
 profile data collection - so you can see where time in going and
 thus to hint at potential optimizations.  The 'spreadsheet.py
 script can be used to view generated profile data.

 See the genhtml/lcov/geninfo man pages for details

 Related options: --parallel, --memory, --profile

In general, the new features and options are implemented uniformly in lcov, genhtml, and geninfo. Most of the features can be enabled/disabled using either command line options or by setting defaults in your 'lcovrc' file. See the lcovrc man page for details.

Finally - and of interest onl to developers - the various lcov-related scripts have been refactored to use a common utility library. See the code in .../lib/lcovutil.pm for details.

Signed-off-by: Henry Cox henry.cox@mediatek.com

[henry2cox]

I just fixed a few typos and updated the PR (force-push - sorry about that). At Peter's request/suggestion, I'm cleaning up a bit so this can be safely merged.

[henry2cox]

Yes…it caused a syntax error on our systems…I just commented it out and didn’t do any more debugging to see why it happened or what functionality I removed. To be honest: I kind of forgot I had done it, and never went back to it.

When I put it back (just now) – I find: @.***

Henry

[oberpar]

Applied as 5f659f63801ef7f94c50a0eb5cffa1ea70f73651 with the following minor changes:

• revert changes to LCOV_PERL_PATH
• revert indentation changes to existing shell scripts
• whitespace fixes
• add subject line
• fixed typos in commit message

Thanks for your contribution and persistence in getting it merged :) And looking forward to a productive cooperation going forward.