search

bazelbuild / continuous-integration

Bazel's Continuous Integration Setup
https://buildkite.com
Apache License 2.0
253 stars 134 forks source link

bazel-at-head-plus-downstream is running Stardoc manual tests which are by design incompatible with Bazel@HEAD #1999

Closed tetromino closed 2 weeks ago

tetromino commented 2 weeks ago

Originally reported at https://github.com/bazelbuild/stardoc/issues/242

For some input .bzl files, Stardoc is expected to produce different output depending on the version of Bazel it runs under (and sometimes - depending on the build flags passed to Bazel). Therefore, it comes with a number of golden tests which are tagged "manual" and which are expected to pass only under a specific version of Bazel with a specific set of flags. Examples: stardoc_self_gen_test, proto_format_test, multiple_files_noenable_bzlmod_test, same_level_file_noenable_bzlmod_test, macro_kwargs_legacy_test, etc. in https://github.com/bazelbuild/stardoc/blob/da374a508b5a5f82e1e00f94889f04f9f86ca1fe/test/BUILD

Stardoc has tasks configured in its presubmit.yml to run these tests with the right versions of Bazel and the right flags. Such tasks have descriptive names such as "Stardoc golden tests requiring Bazel 7.2" and an explicitly set bazel version. Example: https://github.com/bazelbuild/stardoc/blob/da374a508b5a5f82e1e00f94889f04f9f86ca1fe/.bazelci/presubmit.yml#L116

Unfortunately, bazel-at-head-plus-downstream pipeline attempts to run these tasks with Bazel at HEAD, and of course fails.

Possible solutions:

CC @sgowroji

meteorcloudy commented 2 weeks ago

Can you specify skip_in_bazel_downstream_pipeline: "<reason>" for the task?

Example: https://github.com/bazelbuild/rules_foreign_cc/blob/9d5727d5e51bf1be8423dff7996bdb4d847b47e6/.bazelci/config.yaml#L225

tetromino commented 2 weeks ago

I see; and this is documented in https://github.com/bazelbuild/continuous-integration/blob/master/docs/downstream-testing.md#downstream-project-policies

Thanks for the explanation - I think that https://github.com/bazelbuild/stardoc/pull/244 will fix it