taiki-e / cargo-llvm-cov

Cargo subcommand to easily use LLVM source-based code coverage (-C instrument-coverage).
Apache License 2.0
920 stars 57 forks source link

[HELP] error: failed to merge profile data: process didn't exit successfully #345

Closed robertprp closed 8 months ago

robertprp commented 8 months ago

I have been trying to add coverage part to a Rust project that runs tests with nextest. Tests are running correctly but I am not being able to debug the following error:

zero
ci_tests  | ------------
ci_tests  |      Summary [  27.398s] 33 tests run: 33 passed, 1 skipped
ci_tests  | warning: not found *.profraw files in /app/target/llvm-cov-target; this may occur if target directory is accidentally cleared, or running report subcommand without running any tests or binaries
ci_tests  | error: no input files specified. See llvm-profdata merge -help
ci_tests  | error: failed to merge profile data: process didn't exit successfully: `/usr/local/rustup/toolchains/1.70.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-profdata merge -sparse -f /app/target/llvm-cov-target/app-profraw-list -o /app/target/llvm-cov-target/app.profdata` (exit status: 1)
ci_tests  | [cargo-make] ERROR - Error while executing command, exit code: 1
ci_tests  | [cargo-make] WARN - Build Failed.

Simplified docker file:

ARG RUST_VERSION="1.70"
ARG RUST_IMAGE="rust:${RUST_VERSION}-bullseye"

FROM ${RUST_IMAGE} as install
RUN cargo install cargo-chef

FROM install as planner
COPY . .

RUN cargo chef prepare --recipe-path recipe.json

# -- Builder with cache layer --
FROM install as builder
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --workspace --recipe-path recipe.json

COPY . .
RUN cargo build --workspace

# -- Builder with cache layer --
FROM ${RUST_IMAGE} as base
RUN cargo install cargo-make cargo-nextest

RUN rustup toolchain install $RUST_VERSION --component llvm-tools-preview
RUN cargo install cargo-llvm-cov

FROM base as runtime
WORKDIR /app
COPY --from=builder /app/target /app/target
COPY . .
RUN cargo nextest archive --workspace --archive-file test.tar.zst

ENTRYPOINT ["cargo", "make", "docker-ci"]

Command that is run

rm -r /app/target
cargo llvm-cov --json --output-path /app/docker/tests/cov.json nextest --archive-file test.tar.zst --test-threads 1

Versions

cargo-llvm-cov v0.6.4

taiki-e commented 8 months ago
RUN cargo nextest archive --workspace --archive-file test.tar.zst

You have to use cargo llvm-cov nextest-archive, like:

https://github.com/taiki-e/cargo-llvm-cov/blob/aa02184b4c999b05cc17ba7f92e2026d8fc4dc20/.github/workflows/ci.yml#L86-L87

(We should perhaps expand the documentation of nextest to explain this or enhance our own documentation on nextest.)

robertprp commented 8 months ago

The issue with using cargo llvm-cov nextest-archive is that it recompiles everything again.

I have tried without success.

COPY --from=builder /app/target /app/target/llvm-cov-target/
COPY . .
RUN cargo llvm-cov nextest-archive --workspace --archive-file test.tar.zst

ENTRYPOINT ["cargo", "make", "docker-ci"]

Note that i mapped the build to /app/target/llvm-cov-target/ and /app/target/

We use a custom build command for a package and using nextest-archive does throw error Captura de pantalla 2024-01-30 a las 10 47 47

taiki-e commented 8 months ago

The issue with using cargo llvm-cov nextest-archive is that it recompiles everything again.

This is a fundamental limitation of Rust's current code coverage (regardless of which way you use it). There are several RFCs and proposals to improve this, but none have been approved yet.

We use a custom build command for a package and using nextest-archive does throw error

I don't have any information about the project you are building, so I don't know the details, but it is possible that the old build artifact is doing something or the fact that rustflags is being applied to the build script is causing some problems. (The former can be addressed by running cargo clean first, the latter by passing --target option.)

robertprp commented 8 months ago

Thanks for your responses, I will try to investigate a bit more about this topic & avoid recompiling