mozilla / grcov

Rust tool to collect and aggregate code coverage data for multiple source files
Mozilla Public License 2.0
1.18k stars 149 forks source link

all coverage is green, but function % is less than 100% #793

Open spemmons opened 2 years ago

spemmons commented 2 years ago

i have a simple module with tests for each of the functions/methods written, but the report says a percentage are missing -- not sure why -- how can i determine what is considered missing?

marco-c commented 2 years ago

@spemmons can you give a bit more details, maybe an example?

magasek commented 2 years ago

I have seen the same result as @spemmons but for my situation do not consider this an error. In my case this occurred due to #[derive()] statements. My public interfaces include a number of these for the convenience of the crate consumer, but my own code does not use the added traits. Rather than include unit tests for auto-generated traits that I don't use, I use the --excl-line option to exclude the #[derive()] lines from coverage. This means that the Lines of Code covered can reach 100% as long as my unit tests cover every line I actually wrote. But, the function coverage value is less than 100% because the functions defined by the #[derive()] lines are never called.

mickvangelderen commented 1 year ago

I am probably experiencing the same issue:

grcov's html output: image

lcov's html output: image

the code:

fn a() {
    println!("a");
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn it_works() {
        a();
    }
}

commands used to obtain coverage reports:

rm *.profraw
rm -r html/
cargo clean
RUSTFLAGS="-Cinstrument-coverage" LLVM_PROFILE_FILE="default.profraw" cargo test --all-features --no-fail-fast
./grcov . -s . --binary-path target/debug/ -t html --branch --ignore-not-existing
./grcov . -s . --binary-path target/debug/ -t lcov --branch --ignore-not-existing -o target/debug/lcov.info
genhtml -o target/debug/coverage/ --show-details --highlight --ignore-errors source --legend target/debug/lcov.info
qsantos commented 11 months ago

Another case:

use tracing::trace;

fn f() {
    trace!("ok");
}

#[test]
fn test() {
    f();
}

The trace! macro defines a closure that it never calls when tracing is not enabled.

qsantos commented 11 months ago

Another point that can be confusing: --excl-start and --excl-br-start do not change function coverage. As far as I know, there is no way to filter out the unit test functions, short of [no_coverage], which is not yet stable.

In addition, the function coverage as reported by grcov seem to actually be the Instanciation coverage as per Rust documentation of llvm-cov. So if we only run the test1 unit test in the code below, we only get a function coverage of 3 / 6:

I have kept the example short, but note that this also applies to instantiations from other modules, and not just from unit tests.

pub fn a<K>() { 0; }

#[test]
fn test1() {
    a::<u32>();
}

#[test]
fn test2() {
    a::<f32>();
}

At this point, the function coverage is not very useful to me, but I could use an option to disable it, or switch it to function coverage, as defined per Rust documentation of llvm-cov (not instantiation coverage). And maybe some way to exclude unit test functions.