search

bazelbuild / rules_rust

Rust rules for Bazel
https://bazelbuild.github.io/rules_rust/
Apache License 2.0
659 stars 424 forks source link

Code coverage support #690

Open sgammon opened 3 years ago

sgammon commented 3 years ago

Hey authors of rules_rust,

Has there been any consideration yet regarding source coverage with rules_rust? Using projects like cargo-cov, or cargo-tarpaulin, it looks pretty easy from the outside but perhaps not from within Bazel.

IIRC, instrumented_files= would need to be provided for Rust targets, but that part is relatively easy if there is already a tool that can produce lcov-compatible outputs.

sgammon commented 3 years ago

forget i said anything about crates: https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/source-based-code-coverage.html

i've got a sample working locally outside of Bazel, which uses -Zinstrument-coverage/grcov/lcov to generate coverage data and merge with our canonical LCOV report, which already existed. it would be absolutely amazing if this was rolled into rules_rust, though.

i'm happy to contribute if lib maintainers can help me with a direction?

UebelAndre commented 3 years ago

Hey, I'm not super familiar with how coverage works but do think it should be something the rules support.

I'm wondering if other maintainers might be able to shine some light on this?

@hlopko @dfreese pinging you two because you seem to be pretty knowledgeable about some subtle details of Bazel and rustc. Hoping you'll have some thoughts 😄

sluongng commented 3 years ago

Good references should be from rules_go:

Generally Bazel's coverage is not yet complete. If your code coverage tooling does not follow the Java's convention, you gona have a bad time.

hlopko commented 3 years ago

CC @c-mita (I hear they're looking into coverage these days :)

UebelAndre commented 2 years ago

https://github.com/bazelbuild/rules_rust/issues/1077 seems relates

keith commented 2 years ago

I've been looking into this a bit, and it's definitely possible today by using the C++ related coverage script attribute. It feels a bit like a hack but afaict there still no real movement in bazel around this.

UebelAndre commented 2 years ago

I've been looking into this a bit, and it's definitely possible today by using the C++ related coverage script attribute. It feels a bit like a hack but afaict there still no real movement in bazel around this.

@keith What's this coverage script attribute?

keith commented 2 years ago

Here's a super WIP version https://github.com/bazelbuild/rules_rust/pull/1324

hlopko commented 2 years ago

FWIW I think that's our best shot at having working coverage for Rust anytime soon. Took a super-quick look and the PR looks reasonable. Thanks for working on it @keith!

keith commented 2 years ago

Thanks @hlopko !

c-mita commented 2 years ago

I've been looking into this a bit, and it's definitely possible today by using the C++ related coverage script attribute. It feels a bit like a hack but afaict there still no real movement in bazel around this.

I have a personal desire to fix this; to create a general interface for rules to use to support coverage collection (along with cross-language support). But other things have stolen my time.

The _lcov_merger and _collect_cc_coverage attributes are your best bet for now. I suppose technically those form something of an API, but they're completely undocumented.

keith commented 2 years ago

Yea I haven't addressed _lcov_merger, but am using _collect_cc_coverage in https://github.com/bazelbuild/rules_rust/pull/1324, should work fine for now

UebelAndre commented 2 years ago

Does https://github.com/bazelbuild/rules_rust/pull/1324 address the concerns here?

rbtcollins commented 9 months ago

I've been poking at our bazel setup to see why we don't get coverage, and it seems the rules_rust coverage data is incompatible with --experimental_split_coverage_postprocessing - I'm not sure why this should be.

Should I file a separate bug for that, or is this bug being used as a tracking bug for coverage for rust in general?

cristifalcas commented 7 months ago

same for me, it looks like coverage is not created when I have the flag experimental_split_coverage_postprocessing enabled