mozilla / application-services

Firefox Application Services
https://mozilla.github.io/application-services/
Other
608 stars 225 forks source link

Re-enable emission of branch coverage. #3468

Closed thomcc closed 11 months ago

thomcc commented 4 years ago

In https://github.com/mozilla/application-services/pull/3466 we disabled emitting branch coverage info, because in practice the output had too many accuracy issues to be usable. While still not perfect, the line coverage is totally serviceable, and is a much more accurate representation of the actual coverage than the branch info we were getting. See https://github.com/mozilla/application-services/issues/3393 for a description of the most glaring issues we were seeing.

Despite this, we'd still like branch coverage, just... probably only if we have a fix for the issues discussed in #3393.

The ideal solution would be something upstream fixing this for us, e.g. either grcov, codecov.io, or rustc

Alternatively, in the future cargo-tarpaulin could again become an option. Currently it doesn't support branch coverage (it's listed as TODO on their github, but there's no issue to link to, so who knows). From what I know, the approach it uses to compute coverage is very different from what grcov coverage is produced with (-Zprofile, whereas tarpaulin uses ptrace and other OS debugging apis), and so it's possible that it wouldn't have the same issues. This is a stretch, though, and the likely path forward to allowing coverage of Kotlin/Swift depends on using grcov somewhat (but perhaps we could combine the coverage data? Not 100% clear if grcov can take input from something other than .gcno/.gcda files that are output by executing with -Zprofile, would require investigation if we get here).

Either way, at the moment there's not a lot we can do (blocked on upstream fixes or improvements either to grcov, rustc, codecov.io, ...), and the line-based coverage is by all means good-enough, so this is in the backlog for now and the immediate future.

┆Issue is synchronized with this Jira Task ┆Epic: Important backlog

mhammond commented 11 months ago

Moved to bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1865429

Change performed by the Move to Bugzilla add-on.