[WIP] Coverage Analysis

[lght]

This is an attempt to get coverage reporting working for Solaris. Currently available tooling cargo-cov works nicely for generating branch and line-coverage statistics, and getting an HTML-formatted report.

For our purposes, however, we need a format we can upload to Coveralls (for that nice green badge :)

My current working(?) solution is to use llvm-cov gcov to generate .gcov files from .gcda/.gcno files produced by cargo-cov. Ideally this will be one step in the near future.

It is possible to achieve a one-step solution either patching cargo-cov to get intermediate gcov output, or with a script that automates .gcov generation with llvm-cov gcov.

Will update this PR as progress is made.

[lght]

I totally support @tomusdrw idea about autodetecting the available compiler -- as a separate PR we can easily do right now

As do I, this change was a mistake on my part. Reverted the changes :)

the wrteup is pretty solid and easy to understand, let's merge it soon!

Thanks :) I'll keep making changes as I learn more about the available tooling.

Apparently there is an issue with Rust profiling picking up on generic functions (see this issue in cargo-cov). So this approach may work for some basic coverage statistics, but not a long-term / ideal solution by any stretch.

[lght]

I need to get @marco-c and @kennytm a [insert-beverage-of-choice]!

Wouldn't have been able to get this tooling together without their work.

[kirushik]

@lght do you want to wait for a second review, or we can merge this now?

[marco-c]

@lght glad to help!

I've read your guide, are you mixing GCC/LLVM compiled code? https://github.com/marco-c/grcov/issues/25 would make grcov work fine with a mix of GCC/LLVM gcda/gcno files. I haven't prioritized it yet because I don't need it for my purposes, but I can raise its priority if you need it.

Another option would be to split the GCC and LLVM files into two different directories, then run grcov once with the GCC files, once with the LLVM files and once to merge the two.

Note also that grcov doesn't need llvm-cov to be installed, as it is using the LLVM API directly to parse the gcda/gcno files.

[lght]

Another option would be to split the GCC and LLVM files into two different directories, then run grcov once with the GCC files, once with the LLVM files and once to merge the two.

I like this idea a lot. Would make analysis of FFI/Native components easier. Something like "want analysis of wrapped C components, only run GCC grcov pass". Will look into helping with https://github.com/marco-c/grcov/issues/25. Using the headers to determine GCC/LLVM, splitting them inside a tmp working folder, then running the two passes seems like it might work for below feature.

Note also that grcov doesn't need llvm-cov to be installed, as it is using the LLVM API directly to parse the gcda/gcno files.

Did not realize this, makes sense now tho.

are you mixing GCC/LLVM compiled code

Yes. Parity has a number of components that are Rust wrappers around C/C++ code. Is there a way to instrument the build to use all clang?

[marco-c]

Just wanted to let you know that https://github.com/mozilla/grcov/issues/25 has been fixed, so there's no longer any issue with mixing code compiled by GCC and LLVM.