Coverage display on codecov.io does not match locally-generated HTML report

[jwodder]

I don't know whether this is a problem with grcov or Codecov, but you seem more accessible, so I'll start here.

I am attempting to set up coverage reporting of my Rust project using grcov and codecov.io. When I generate HTML coverage reports locally like so:

export RUSTFLAGS="${RUSTFLAGS:+$RUSTFLAGS }-Cinstrument-coverage"
cargo build
export LLVM_PROFILE_FILE="target/coverage/prof/%p-%m.profraw"
cargo test

grcov \
    --source-dir . \
    --binary-path target/debug \
    --branch \
    --excl-start 'mod tests \{' \
    --ignore 'tests/*' \
    -t html \
    -o target/coverage/html \
    target/coverage/prof

then the coverage is shown as 88.61%, and the report (mostly) makes sense. However, when I upload a report to codecov from GitHub Actions with the following workflow:

name: Test

on:
  push:
  pull_request:
  schedule:
    - cron: '0 0 1 * *'

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        toolchain:
          - '1.60'
          - stable
          - beta
          - nightly
    steps:
      - name: Check out repository
        uses: actions/checkout@v3

      - name: Configure Git identity
        # Needed for a couple tests
        run: |
          git config --global user.name 'GitHub Actions'
          git config --global user.email 'ghactions@nil.nil'

      - name: Install Rust
        run: |
          rustup update ${{ matrix.toolchain }}
          rustup default ${{ matrix.toolchain }}
          rustup component add llvm-tools-preview
          cargo install grcov

      - name: Build crate
        run: cargo build --verbose
        env:
          RUSTFLAGS: "-Cinstrument-coverage"

      - name: Test crate
        run: cargo test --verbose
        env:
          LLVM_PROFILE_FILE: "target/coverage/prof/%p-%m.profraw"
          RUSTFLAGS: "-Cinstrument-coverage"

      - name: Create coverage report
        run: |
         grcov \
           --source-dir . \
           --binary-path target/debug \
           --branch \
           --excl-start 'mod tests \{' \
           --ignore 'tests/*' \
           -t lcov \
           -o lcov.info \
           target/coverage/prof

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v3
        with:
          files: lcov.info
          fail_ci_if_error: false

then the coverage is shown as 48.21%, and much of the report does not match the HTML report, nor does it make much sense.

For example, compare this section of the HTML report:

with the corresponding section in codecov's report:

Another example: In the HTML report, all of the lines shown below are neither hit nor missed, yet codecov shows a mixture of hit & missed lines, including some of both in the comments!

In addition, I previously had my tests in my src/lib.rs file but later moved them to tests/. When they were in src/lib.rs, I was able to get the HTML report to show the tests as neither hit nor missed, yet Codecov showed the tests as largely missed, with some hits thrown in.

You can find the report uploaded to codecov for the "stable" run of this commit here: https://github.com/mozilla/grcov/files/9072729/build.txt.

[erhanbaris]

I think I found the issue. grcov generate coverage information about dependencies and that information confuse the codecov. You can find my solution at the following link: https://github.com/erhanbaris/yummy/blob/main/.github/workflows/rust.yml Also here is the command:

grcov . \
            --binary-path ./target/debug/deps/ \
            --source-dir . \
            --excl-start 'mod test* \{' \
            --ignore 'tests/*' \
            --ignore "*test.rs" \
            --ignore "*tests.rs" \
            --ignore "*github.com*" \
            --ignore "*libcore*" \
            --ignore "*rustc*" \
            --ignore "*liballoc*" \
            --ignore "*cargo*" \
            -t coveralls+ \
            --token ? \
            -o ./cov_normal.json

Codecov shows correct coverage information now. I hope that information will help you.