Merge coverage from external tests running on napi-rs node binding

[archfz]

I am trying to merge coverage from rust unit test with integration test that are run upon napi-rs generated node binding. I am trying to do this based on Get coverage of external tests section from README.

Once tests run on the node binding I do get the .profraw file generated, but once I run cargo llvm-cov --html I get empty coverage report.

The documentation says that binaries need to be ran from under ./target/debug. But the node binary / binding is actually in the root of the project. Not sure if this is the problem.

So I am not sure if this is not supported, I am doing something wrong or there is some bug.

[archfz]

Small correction. I get the above empty result when I only run the integration tests (when I don't have .profraw files from cargo unit test). With unit tests as well I get the same coverage as without integration tests. So basically the coverage data from integration tests is not processable or mappable somehow.

I can provide the .profraw if need.

[taiki-e]

Could you be more specific about the commands you have actually tried?

cargo llvm-cov --html

Basically, in the context of "Get coverage of external tests" section, you need to call cargo, not cargo llvm-cov, unless the clean subcommand or the --no-report--no-run flag is used. (as the example in that section does -- though we should have a comment explaining it)

[archfz]

I am running these commands:

source <(cargo llvm-cov show-env --export-prefix)
napi build --platform # this runs rust build behind
cargo llvm-cov clean --workspace
npm run test:js # this runs test upon node module built with rs
cargo llvm-cov --no-run --html

I am not sure what you mean by you need to call cargo, not cargo llvm-cov

[taiki-e]

napi build --platform # this runs rust build behind
cargo llvm-cov clean --workspace

These two probably need to be swapped.

[archfz]

I get the same results. Clean seems to remove only coverage related data, not build related data.

[archfz]

Do you know how to debug this? Or should I create a reproducible repo for this?

[taiki-e]

Do you know how to debug this? Or should I create a reproducible repo for this?

A small reproduction would be great.

[archfz]

@taiki-e Here it is https://github.com/archfz/cargo-llvm-cov-issues-207

[taiki-e]

Thanks for the repro. In my environment (aarch64 macos), the problem seems to be both of the following:

• this cargo llvm-cov clean --workspace should be called before this yarn build.
• this source <(cargo llvm-cov show-env --export-prefix) fails due to broken pipe error. (i.e., env vars are not set)

And, in my environment, the following patch fixed the problem.

diff --git a/coverage.sh b/coverage.sh
index e9fa652..722bf98 100755
--- a/coverage.sh
+++ b/coverage.sh
@@ -3,7 +3,7 @@
 set -e

 source <(cargo llvm-cov show-env --export-prefix)
-yarn build
 cargo llvm-cov clean --workspace
+yarn build
 yarn test:js
 cargo llvm-cov --no-run --html -v
diff --git a/package.json b/package.json
index 38d8151..5cf1dbd 100644
--- a/package.json
+++ b/package.json
@@ -24,7 +24,7 @@
   "scripts": {
     "build": "napi build --platform",
     "test:js": "ava",
-    "coverage:external": "./coverage.sh",
+    "coverage:external": "bash coverage.sh",
     "coverage:internal": "cargo llvm-cov --html"
   },
   "packageManager": "yarn@3.2.1"

[archfz]

@taiki-e Thank you. That worked.