bazelbuild / rules_swift

Bazel rules to build Swift on Apple and Linux platforms
Apache License 2.0
312 stars 137 forks source link

Does rules_swift support baseline coverage? #828

Open gabrielrussoc opened 2 years ago

gabrielrussoc commented 2 years ago

Hey,

I'm trying to implement baseline coverage on scala rules and I was looking for some real world examples. Do you support baseline coverage? A quick github search didn't yield any results.

https://bazel.build/contribute/codebase#coverage-collection

Another core concept is baseline coverage. This is the coverage of a library, binary, or test if no code in it was run. The problem it solves is that if you want to compute the test coverage for a binary, it is not enough to merge the coverage of all of the tests because there may be code in the binary that is not linked into any test. Therefore, what we do is to emit a coverage file for every binary which contains only the files we collect coverage for with no covered lines. The baseline coverage file for a target is at bazel-testlogs/$PACKAGE/$TARGET/baseline_coverage.dat . It is also generated for binaries and libraries in addition to tests if you pass the --nobuild_tests_only flag to Bazel.

keith commented 2 years ago

As far as I understand it baseline coverage isn't actually supported in bazel, but is probably half implemented because it's missing pieces from blaze. Later in the doc you linked it says:

Baseline coverage is currently broken

gabrielrussoc commented 2 years ago

@keith https://github.com/bazelbuild/bazel/issues/5716 makes me think it might only be broken for native rules like C++ and Java, but perhaps starlark rules could work.

Anyway, is there any workaround? If you delete all your _test targets, does bazel coverage //... produce an empty report?

keith commented 2 years ago

Right now coverage is only produced when tests are run, so I guess what you'd have to do instead is launch the binary collecting coverage, kill it, and then run tests with coverage. Which we definitely don't handle.