Open niebayes opened 1 day ago
Hmm, I have a feeling it is something related to the propagation of environment variables, but it doesn't seem to be an obvious problem that I can figure out just by reading the description. Could you provide a reproduction? As always, it is difficult to investigate an issue unless I can reproduce it on my end.
@taiki-e Very appreciate for your reply! Since our project is commercial and not open-sourced, I cannot provide a direct reproduction. However, I think I could make a toy project while enough for reproduction. I will come back when the toy project is ready and I can reproduce the issue with the toy project.
Situation 1: We embed cargo-llvm-cov into a docker image and use it as the base image for running CI. However, only invoked with
cargo llvm-cov
, the coverage is correct. If we instead build the binary withcargo build ...
and then perform testing with./target/debug/<binary> ...
, the coverage is not correct, specifically almost all lines are missed. Note, we have checked the README and properly set the environment variables withsource <(cargo llvm-cov show-env --export-prefix)
.Situation 2: We also use a docker image as the base image for running CI, however, with cargo-llvm-cov not pre-installed. Instead, each CI job installs cargo-llvm-cov on its own. This time, astonishingly, the generated coverage is correct even if some tests are not invoked with
cargo llvm-cov
.Additional context:
cargo install cargo-llvm-cov
and by installingllvm-tools-preview
withrustup component add llvm-tools-preview
.