fmeum / with_cfg.bzl

Apache License 2.0
40 stars 2 forks source link

Does not work on test_suite #83

Closed matthewjh closed 2 months ago

matthewjh commented 5 months ago

Thanks @fmeum, really enjoying this package so far. I am trying to use it on test_suite as I want to run the "same" tests under different configs.

Given...

_tsc_test_suite, _tsc_test_suite_internal = with_cfg(native.test_suite).set(Label("//tools/ts-project:flag-use-tsc"), True)).build()

and...

_tsc_test_suite(
      name = "aot_test",
      tests = [
        tests_name
     ]
)

This happens:

bazel test //libs/twi-generic:aot_test
ERROR: libs/twi-generic/BUILD:5:13: in _tsc_test_suite_internal rule //libs/twi-generic:aot_test_with_cfg: non-test target '//libs/twi-generic:aot_test_with_cfg' depends on testonly target '//libs/twi-generic:aot_test_/aot_test' and doesn't have testonly attribute set
ERROR: libs/twi-generic/BUILD:5:13: Analysis of target '//libs/twi-generic:aot_test_with_cfg' failed
ERROR: Analysis of target '//libs/twi-generic:aot_test' failed; build aborted: Analysis failed
INFO: Elapsed time: 2.551s, Critical Path: 0.00s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
ERROR: No test targets were found, yet testing was requested
fmeum commented 5 months ago

While I can fix that particular error, I found that a wrapped test_suite no longer runs any tests. This isn't too surprising as test_suite is a deeply magical built-in rule and Bazel doesn't really support targets that depend on tests.

Could you instead try to transition the individual tests?

fmeum commented 5 months ago

This may be doable after https://github.com/bazelbuild/bazel/issues/14993.

fmeum commented 3 months ago

This can be worked around by transitioning the individual tests instead and wrapping them into a test_suite via a helper macro, see https://github.com/fmeum/with_cfg.bzl/tree/main/examples/java_test_suite_multiple_jdks for an example.

fmeum commented 2 months ago

I will close this as "not planned" since I don't think there is anything that can be done without Bazel changes. The test I linked documents the best workaround I'm aware of.