Cargo subcommand to easily use LLVM source-based code coverage.
This is a wrapper around rustc -C instrument-coverage
and provides:
cargo test
, cargo run
, and cargo nextest
with command-line interface compatible with cargo.Table of Contents:
By default, run tests (via cargo test
), and print the coverage summary to stdout.
cargo llvm-cov
Currently, doc tests are disabled by default because nightly-only features are required to make coverage work for doc tests. see #2 for more.
To run cargo run
instead of cargo test
, use run
subcommand.
cargo llvm-cov run
With html report (the report will be generated to target/llvm-cov/html
directory):
cargo llvm-cov --html
open target/llvm-cov/html/index.html
or
cargo llvm-cov --open
With plain text report (if --output-path
is not specified, the report will be printed to stdout):
cargo llvm-cov --text | less -R
With json report (if --output-path
is not specified, the report will be printed to stdout):
cargo llvm-cov --json --output-path cov.json
With lcov report (if --output-path
is not specified, the report will be printed to stdout):
cargo llvm-cov --lcov --output-path lcov.info
You can get a coverage report in a different format based on the results of a previous run by using cargo llvm-cov report
.
cargo llvm-cov --html # run tests and generate html report
cargo llvm-cov report --lcov # generate lcov report
cargo llvm-cov
/cargo llvm-cov run
/cargo llvm-cov nextest
cleans some build artifacts by default to avoid false positives/false negatives due to old build artifacts.
This behavior is disabled when --no-clean
, --no-report
, or --no-run
is passed, and old build artifacts are retained.
When using these flags, it is recommended to first run cargo llvm-cov clean --workspace
to remove artifacts that may affect the coverage results.
cargo llvm-cov clean --workspace # remove artifacts that may affect the coverage results
cargo llvm-cov --no-clean
You can merge the coverages generated under different test conditions by using --no-report
and cargo llvm-cov report
.
cargo llvm-cov clean --workspace # remove artifacts that may affect the coverage results
cargo llvm-cov --no-report --features a
cargo llvm-cov --no-report --features b
cargo llvm-cov report --lcov # generate report without tests
Note: To include coverage for doctests you also need to pass --doctests
to cargo llvm-cov report
.
Set CC
, CXX
, LLVM_COV
, and LLVM_PROFDATA
environment variables to Clang/LLVM compatible with the LLVM version used in rustc, and run cargo-llvm-cov with --include-ffi
flag.
CC=<clang-path> \
CXX=<clang++-path> \
LLVM_COV=<llvm-cov-path> \
LLVM_PROFDATA=<llvm-profdata-path> \
cargo llvm-cov --lcov --include-ffi
cargo test
, cargo run
, and cargo nextest
are available as builtin, but cargo-llvm-cov can also be used for arbitrary binaries built using cargo (including other cargo subcommands or external tests that use make, xtask, etc.)
# Set the environment variables needed to get coverage.
source <(cargo llvm-cov show-env --export-prefix)
# Remove artifacts that may affect the coverage results.
# This command should be called after show-env.
cargo llvm-cov clean --workspace
# Above two commands should be called before build binaries.
cargo build # Build rust binaries.
# Commands using binaries in target/debug/*, including `cargo test` and other cargo subcommands.
# ...
cargo llvm-cov report --lcov # Generate report without tests.
Note: cargo-llvm-cov subcommands other than report
and clean
may not work correctly in the context where environment variables are set by show-env
; consider using normal cargo
/cargo-nextest
commands.
Note: To include coverage for doctests you also need to pass --doctests
to both cargo llvm-cov show-env
and cargo llvm-cov report
.
To exclude specific file patterns from the report, use the --ignore-filename-regex
option.
cargo llvm-cov --open --ignore-filename-regex build
To exclude the specific function from coverage, use the #[coverage(off)]
attribute.
Since #[coverage(off)]
is unstable, it is recommended to use it together with cfg(coverage)
or cfg(coverage_nightly)
set by cargo-llvm-cov.
#![cfg_attr(coverage_nightly, feature(coverage_attribute))]
#[cfg_attr(coverage_nightly, coverage(off))]
fn exclude_from_coverage() {
// ...
}
cfgs are set under the following conditions:
cfg(coverage)
is always set when using cargo-llvm-cov (unless --no-cfg-coverage
flag passed)cfg(coverage_nightly)
is set when using cargo-llvm-cov with nightly toolchain (unless --no-cfg-coverage-nightly
flag passed)If you want to ignore all #[test]
-related code, consider using coverage-helper crate version 0.2+.
cargo-llvm-cov excludes code contained in the directory named tests
from the report by default, so you can also use it instead of coverage-helper crate.
Note: #[coverage(off)]
was previously named #[no_coverage]
. When using #[no_coverage]
in the old nightly, replace feature(coverage_attribute)
with feature(no_coverage)
, coverage(off)
with no_coverage
, and coverage-helper
0.2+ with coverage-helper
0.1.
Here is an example of GitHub Actions workflow that uploads coverage to Codecov.
name: Coverage
on: [pull_request, push]
jobs:
coverage:
runs-on: ubuntu-latest
env:
CARGO_TERM_COLOR: always
steps:
- uses: actions/checkout@v4
- name: Install Rust
run: rustup update stable
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Generate code coverage
run: cargo llvm-cov --all-features --workspace --lcov --output-path lcov.info
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos
files: lcov.info
fail_ci_if_error: true
Currently, when using --lcov
flag, only line coverage is available on Codecov.
By using --codecov
flag instead of --lcov
flag, you can use region coverage on Codecov:
- name: Generate code coverage
run: cargo llvm-cov --all-features --workspace --codecov --output-path codecov.json
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos
files: codecov.json
fail_ci_if_error: true
Note that the way Codecov shows region/branch coverage is not very good.
You can display coverage in VS Code using Coverage Gutters.
Coverage Gutters supports lcov style coverage file and detects lcov.info
files at the top level or in the coverage
directory. Below is an example command to generate the coverage file.
cargo llvm-cov --lcov --output-path lcov.info
You may need to click the "Watch" label in the bottom bar of VS Code to display coverage.
You can override these environment variables to change cargo-llvm-cov's behavior on your system:
CARGO_LLVM_COV_TARGET_DIR
-- Location of where to place all generated artifacts, relative to the current working directory. Default to <cargo_target_dir>/llvm-cov-target
.CARGO_LLVM_COV_SETUP
-- Control behavior if llvm-tools-preview
component is not installed. See #219 for more.LLVM_COV
-- Override the path to llvm-cov
. You may need to specify both this and LLVM_PROFDATA
environment variables if you are using --include-ffi
flag or if you are using a toolchain installed without via rustup. llvm-cov
version must be compatible with the LLVM version used in rustc.LLVM_PROFDATA
-- Override the path to llvm-profdata
. See LLVM_COV
environment variable for more.LLVM_COV_FLAGS
-- A space-separated list of additional flags to pass to all llvm-cov
invocations that cargo-llvm-cov performs. See LLVM documentation for available options.LLVM_PROFDATA_FLAGS
-- A space-separated list of additional flags to pass to all llvm-profdata
invocations that cargo-llvm-cov performs. See LLVM documentation for available options.LLVM_PROFILE_FILE_NAME
-- Override the file name (the final component of the path) of the LLVM_PROFILE_FILE
. See LLVM documentation for available syntax.See also environment variables that Cargo reads. cargo-llvm-cov respects many of them.
If JSON is selected as output format (with the --json
flag), then cargo-llvm-cov will add additional contextual information at the root of the llvm-cov data. This can be helpful for programs that rely on the output of cargo-llvm-cov.
{
// Other regular llvm-cov fields ...
"cargo_llvm_cov": {
"version": "0.0.0",
"manifest_path": "/path/to/your/project/Cargo.toml"
}
}
version
specifies the version of cargo-llvm-cov that was used. This allows other programs to verify a certain version of it was used and make assertions of its behavior.manifest_path
defines the absolute path to the Rust project's Cargo.toml that cargo-llvm-cov was executed on. It can help to avoid repeating the same option on both programs.For example, when forwarding the JSON output directly to another program:
cargo-llvm-cov --json | some-program
cargo +stable install cargo-llvm-cov --locked
Currently, installing cargo-llvm-cov requires rustc 1.73+.
cargo-llvm-cov is usually runnable with Cargo versions older than the Rust version
required for installation (e.g., cargo +1.60 llvm-cov
). Currently, to run
cargo-llvm-cov requires Cargo 1.60+.
You can download prebuilt binaries from the Release page. Prebuilt binaries are available for macOS, Linux (gnu and musl), and Windows (static executable).
You can use taiki-e/install-action to install prebuilt binaries on Linux, macOS, and Windows. This makes the installation faster and may avoid the impact of problems caused by upstream changes.
- uses: taiki-e/install-action@cargo-llvm-cov
When used with nextest:
- uses: taiki-e/install-action@cargo-llvm-cov
- uses: taiki-e/install-action@nextest
You can install cargo-llvm-cov from the Homebrew tap maintained by us (x86_64/aarch64 macOS, x86_64/aarch64 Linux):
brew install taiki-e/tap/cargo-llvm-cov
Alternatively, you can install cargo-llvm-cov from homebrew-core (x86_64/aarch64 macOS, x86_64 Linux):
brew install cargo-llvm-cov
You can install cargo-llvm-cov from the Scoop bucket maintained by us:
scoop bucket add taiki-e https://github.com/taiki-e/scoop-bucket
scoop install cargo-llvm-cov
You can install cargo-llvm-cov using cargo-binstall:
cargo binstall cargo-llvm-cov
You can install cargo-llvm-cov from the extra repository:
pacman -S cargo-llvm-cov
See also the code-coverage-related issues reported in rust-lang/rust.
-Z minimal-versions
.Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.