[Bazel CI] Test docs:are_docs_up_to_date_test is failing in Bazel@HEAD

[sgowroji]

CI: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3920#01900fc7-e6cb-4aaa-bf71-2c27285f1a16

Platform: Ubuntu

Logs:

(04:23:42) ERROR: /var/lib/buildkite-agent/builds/bk-docker-cdhd/bazel-org-repo-root/rules_kotlin/kotlin/BUILD:39:12: Generating proto for Starlark doc for jvm failed: (Exit 1): jvm_stardoc failed: error executing Stardoc command (from target //kotlin:jvm)
--
  | (cd /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f4408b6fdca7fe1c80b66318279eee19/sandbox/linux-sandbox/268/execroot/rules_kotlin && \
  | exec env - \
  | bazel-out/k8-opt-exec-ST-a828a81199fe/bin/kotlin/jvm_stardoc '--input=@//kotlin:jvm.bzl' '--workspace_name=rules_kotlin' '--output=bazel-out/k8-fastbuild/bin/kotlin/jvm.raw')
  | # Configuration: 3aaf33ed4547995c2a1ab6a3e02df7e4379d9da6983c62d312448d7dc71fefec
  | # Execution platform: @@internal_platforms_do_not_use//host:host
  |  
  | Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
  | com.google.devtools.build.skydoc.SkydocMain$StarlarkEvaluationException: File /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f4408b6fdca7fe1c80b66318279eee19/sandbox/linux-sandbox/268/execroot/rules_kotlin/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/kotlin/jvm_stardoc.runfiles/rules_java_builtin/toolchains/toolchain_utils.bzl imported '//java/common:java_common.bzl', yet /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f4408b6fdca7fe1c80b66318279eee19/sandbox/linux-sandbox/268/execroot/rules_kotlin/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/kotlin/jvm_stardoc.runfiles/rules_java_builtin/java/common/java_common.bzl was not found.
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:454)
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
  | at com.google.devtools.build.skydoc.SkydocMain.eval(SkydocMain.java:289)
  | at com.google.devtools.build.skydoc.SkydocMain.main(SkydocMain.java:151)
  | Caused by: java.nio.file.NoSuchFileException: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f4408b6fdca7fe1c80b66318279eee19/sandbox/linux-sandbox/268/execroot/rules_kotlin/bazel-out/k8-opt-exec-ST-a828a81199fe/bin/kotlin/jvm_stardoc.runfiles/rules_java_builtin/java/common/java_common.bzl
  | at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
  | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
  | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
  | at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
  | at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
  | at java.base/java.nio.file.Files.newByteChannel(Files.java:422)
  | at java.base/java.nio.file.Files.readAllBytes(Files.java:3206)
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:429)
  | at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
  | ... 7 more
//docs:are_docs_up_to_date_test                                 FAILED TO BUILD
--

Culprit:

Steps:

 git clone https://github.com/bazelbuild/rules_kotlin
git reset 7dcb7f94f3f367110d75a3ea4464ae4e4cbbf8f0  --hard
export USE_BAZEL_VERSION=e73dca7d03e019173e051b2b05a7404ba89e6912
bazel test //:all_tests 

CC Greenteam @salmasamy

[corbinrsmith]

Hm. Interesting. I guess java_common moved.

Will sort it out.

[sgowroji]

CC @corbinrsmith Any update on the above issue ?

[restingbull]

Not yet. From what I can tell rules_kotlin is relying on the rules_java bzl_library, which should have the file. Digging into the build graph.

[sgowroji]

@restingbull  Any update?

[restingbull]

Passes on MacOS. Debugging on Ubuntu.

[restingbull]

Updating bzlmod to 7.6.5 solves it with bzlmod -- still fails on workspace.

[restingbull]

The problem appears to be coming from @bazel_tools//tools/build_defs/repo:bzl_srcs, which is loaded to provide http_archive, http_file definitions when generating docs.

export USE_BAZEL_VERSION=880c17c67a9b4e17e9753a5c6e2a759bff5a2cfe; bazel query --enable_bzlmod=false 'allpaths("//kotlin:stardoc", "@rules_java_builtin//toolchains:toolchain_utils.bzl")'
//kotlin:core
//kotlin:core_stardoc
//kotlin:js
//kotlin:js_stardoc
//kotlin:jvm
//kotlin:jvm_stardoc
//kotlin:lint
//kotlin:lint_stardoc
//kotlin:repositories.doc
//kotlin:repositories.doc_stardoc
//kotlin:stardoc
//kotlin/internal:internal
//kotlin/internal/jvm:jvm
//third_party:bzl
@bazel_tools//tools:bzl_srcs
@bazel_tools//tools/jdk:bzl_srcs
@rules_java_builtin//toolchains:bzl_srcs
@rules_java_builtin//toolchains:toolchain_utils.bzl

[restingbull]

Adding

bzl_library(
    name = "bzl",
    srcs = [
        "jarjar.bzl",
        "@bazel_tools//tools:bzl_srcs",
    ],
    deps = [
        "@rules_java_builtin//java/common",
    ],
)

Fixes it without bzlmod, but breaks it with -- the native java rules rewrite rules_java to rules_java_builtin.

Paths forward:

• Stub out http_archive/http_file for doc generation.
• Conditionally add rules_java_builtin if runnign with bzlmod.

[restingbull]

Ah, wrong path. While close (the sources are pulled from bazel_tools repository), the issue is in importing the jdk toolchain utils, which moved to rules_java.

Requires bumping rules_java to 7.6.5, which is not compatible with bazel 6.

[restingbull]

@Bencodes / @nkoroste / @oliviernotteghem : Fixing this will make rules_kotlin incompatible with java 6.

[restingbull]

See https://github.com/bazelbuild/rules_kotlin/pull/1190

[restingbull]

@Bencodes / @nkoroste / @oliviernotteghem : Fixing this will make rules_kotlin incompatible with java 6.

Never mind -- just not using the utility functions turns out to be a better approach.