xd009642 / tarpaulin

A code coverage tool for Rust projects
https://crates.io/crates/cargo-tarpaulin
Apache License 2.0
2.5k stars 180 forks source link

Line coverage false negative issue with match in generic functions and multi-line block #1078

Open Fran314 opened 2 years ago

Fran314 commented 2 years ago

Describe the bug Tarpaulin doesn't recognise some lines as covered even though they are. In particular, in the example tarpaulin says that line 10 and 16 (just those lines, not the whole blocks) are not covered.

This does not happen if the match branches are oneline (that is, if instead of

Ok(()) => {
    //
    Ok(())
}

I use Ok(()) => Ok(()) the line is marked as covered)

This does not happen also if the function that wraps the match does not use a generic type: the two functions in the example do_nothing_generic and do_nothing_str are practically the same, the only exception being that one takes a type that implements to_string() while the other takes a &str, but the bug happens only in the function that takes a generic type.

To Reproduce [./Cargo.toml]

[package]
name = "tarpaulin-bug-match"
version = "0.1.0"
edition = "2021"

[dependencies]

[./src/lib.rs]

fn is_boom(path: String) -> Result<(), ()> {
    if path == "BOOM".to_string() {
        Ok(())
    } else {
        Err(())
    }
}
pub fn do_nothing_generic<P: std::string::ToString>(path: P) -> Result<(), ()> {
    match is_boom(path.to_string()) {
        Ok(()) => {
            // This comment needs to be here for the bug to happen.
            // More precisely, it needs not to be Ok(()) => Ok(()), and if I don't
            //  put a comment here, my IDE autoformats it to Ok(()) => Ok(())
            Ok(())
        }
        Err(()) => {
            // Same as above
            Err(())
        }
    }
}

pub fn do_nothing_str(path: &str) -> Result<(), ()> {
    match is_boom(path.to_string()) {
        Ok(()) => {
            // Same as above, but here it doesn't reproduce the bug
            Ok(())
        }
        Err(()) => {
            // Same as above, but here it doesn't reproduce the bug
            Err(())
        }
    }
}

#[cfg(test)]
mod tests {
    use super::{do_nothing_generic, do_nothing_str};

    #[test]
    fn test() {
        assert!(do_nothing_str("BOOM").is_ok());
        assert!(do_nothing_str("not boom").is_err());

        assert!(do_nothing_generic("BOOM").is_ok());
        assert!(do_nothing_generic("not boom").is_err());
    }
}

On calling cargo tarpaulin the output is the following

Aug 21 14:46:36.335  INFO cargo_tarpaulin::config: Creating config
Aug 21 14:46:36.353  INFO cargo_tarpaulin: Running Tarpaulin
Aug 21 14:46:36.353  INFO cargo_tarpaulin: Building project
Aug 21 14:46:36.353  INFO cargo_tarpaulin::cargo: Cleaning project
   Compiling tarpaulin-bug-match v0.1.0 (/home/[redacted, path to working directory]/tarpaulin-bug-match)
    Finished test [unoptimized + debuginfo] target(s) in 0.35s
  Executable unittests src/lib.rs (target/debug/deps/tarpaulin_bug_match-6dc94afcf7beb30b)
Aug 21 14:46:36.764  INFO cargo_tarpaulin::process_handling::linux: Launching test
Aug 21 14:46:36.764  INFO cargo_tarpaulin::process_handling: running /home/[redacted, path to working directory]/tarpaulin-bug-match/target/debug/deps/tarpaulin_bug_match-6dc94afcf7beb30b

running 1 test
test tests::test ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

Aug 21 14:46:36.953  INFO cargo_tarpaulin::report: Coverage Results:
|| Uncovered Lines:
|| src/lib.rs: 10, 16
|| Tested/Total Lines:
|| src/lib.rs: 17/19 +0.00%
|| 
89.47% coverage, 17/19 lines covered, +0% change in coverage

I'm reproducing this on Linux Mint 20.3, and this is the output of uname -a

Linux lenovo-ideapad5 5.15.0-46-generic #49~20.04.1-Ubuntu SMP Thu Aug 4 19:15:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

The version of rustc used is rustc 1.61.0 (fe5b13d68 2022-05-18)

Expected behavior The lines 10, 16 should be counted as covered, so the Total Lines for src/lib.rs should be 19/19, and the coverage should be 100% instead of 89.47%

qsantos commented 10 months ago

I have encountered the same issue, as well as similar ones, for which I was able to extract minimal working examples. I have the feeling that Tarpaulin just get very confused with generics.

struct S1 { v: Option<i32>, }
fn f1<T>() {
    let s = S1 { v: Some(0) };
    Box::new(S1 {
        v: s
            .v
            .map(|v| 42),
    });
}
#[test]
fn test1() { f1::<()>(); }

struct S2 { u: i32, }
fn f2<T>() {
    Box::new(S2 {
        u: 0,
    });
}
#[test]
fn test2() { f2::<()>(); }

fn f3<T>() {
    Some(0)
    .map(
        |
        v
        |
        42
    );
}
#[test]
fn test3() { f3::<()>(); }

Note that Box is needed, probably to avoid optimization that will remove all code. I get:

image

qsantos commented 10 months ago

Note we can just use 'a instead of T:

struct S1 { v: Option<i32>, }
fn f1<'a>() {
    let s = S1 { v: Some(0) };
    Box::new(S1 {
        v: s
            .v
            .map(|v| 42),
    });
}
#[test]
fn test1() { f1(); }

struct S2 { u: i32, }
fn f2<'a>() {
    Box::new(S2 {
        u: 0,
    });
}
#[test]
fn test2() { f2(); }

fn f3<'a>() {
    Some(0)
    .map(
        |
        v
        |
        42
    );
}
#[test]
fn test3() { f3(); }

gets me

image

qsantos commented 10 months ago

Here is another MWE with --engine llvm (the 0; is also marked as not covered without --engine llvm):

fn f<'a>() {
    let a = if true { 0 } else { 0 };
    0;
}

#[test]
fn test() {
    f();
}

image

thetayloredman commented 10 months ago

I'm seeing this in my project zirco-lang/zrc as well: image

The image doesn't show it well, but the two matchers are not covered