Closed kdarkhan closed 7 months ago
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
Thanks for integrating Codecov - We've got you covered :open_umbrella:
Given the above limitations, I propose using
--codecov
based approach which this PR implements.
Excellent, this looks great! I completely agree that we should just use whatever format is easiest :smile:
cargo-llvm-cov is used for generating coverage reports.
The library supports different report types which all have their own pros and cons:
--lcov
- reports inlcov
format which are supported by Codecov. They support only line level coverage and will not provide any info on regions. Example report can be seen here.--codecov
- Codecov's custom format which somewhat supports region coverage but not fully. Example report can be seen here. Notice that some lines are colored in yellow - which means they are partially covered but exact regions within lines are not highlighted.--html
- html output which is provided byllvm-cov
. This output is the most informative as it highlights covered and uncovered regions. I pushed an example of this to Github Pages here. Unfortunately, Codecov does not support such outputs. We have an option to push these reports manually to some Github Pages repo during PR review. Or we can host these assets in Github Artifacts (GA). GA, however, will not make it easy to preview generated HTML - users will have to download the assets and open locally. More convenient preview in GA is blocked by this issue.Given the above limitations, I propose using
--codecov
based approach which this PR implements.I am not integrating reports provided by
cargo fuzz coverage
into this yet and planning to do it in a separate PR once we agree on the initial approach.cargo fuzz coverage
generates an output which is closer to--lcov
mentioned above. There is a hack that I think can be implemented to make it work with--codecov
reports.