Bazel crashes using external aspect + `--noenable_workspace`

lbcjbb commented 3 weeks ago

Description of the bug:

With bzlmod and the --noenable_workspace option, Bazel crashes when we try to use an external aspect defined by the --aspects and --override_repository options.

$ bazel build \
   --enable_bzlmod \
   --noenable_workspace \
   --aspects=@@foo//:foo.bzl%foo_aspect \
   --override_repository=foo=%workspace%/foo \
   //...
Analyzing: target //:dummy (1 packages loaded, 0 targets configured)
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.IllegalStateException: Evaluation of SkyKey failed and no dependencies were requested: KeyForBuild{label=@@foo//:foo.bzl, isBuildPrelude=false} IncrementalInMemoryNodeEntry{key=KeyForBuild{label=@@foo//:foo.bzl, isBuildPrelude=false}, value=null, dirtyBuildingState=InitialBuildingState{dirtyState=REBUILDING, signaledDeps=5, externalDeps=0, dirtyDirectDepIndex=0}, version=com.google.devtools.build.skyframe.Version$MinimalVersion@5937f9c0, directDeps=GroupedDeps{size=5, elements=[Starlark @_builtins, CONTAINING_PACKAGE_LOOKUP:@@foo//, FILE:[<output_base>/external/foo]/[foo.bzl], PRECOMPUTED:starlark_semantics, Key{repoName=@@foo, rootModuleShouldSeeWorkspaceRepos=true}]}, reverseDeps=ReverseDeps{reverseDeps=[], singleReverseDep=false, dataToConsolidate=[BuildTopLevelAspectsDetailsKey{topLevelAspectsClasses=[@@foo//:foo.bzl%foo_aspect], topLevelAspectsParameters={}}]}}
    at com.google.common.base.Preconditions.checkState(Preconditions.java:835)
    at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:690)
    at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:414)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

No problem with --enable_workspace option.
No problem when passing a missing bzl file in the --aspects option, e.g --aspects=@@foo//:xxx.bzl%foo_aspect.
Same problem when passing a non exported symbol in the --aspects option, e.g --aspects=@@foo//:foo.bzl%xxx.
Same problem with Bazel 7.1.2.

Which category does this issue belong to?

Core, External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

.
├── .bazelversion
├── BUILD.bazel
├── foo
│   ├── BUILD.bazel
│   ├── foo.bzl
│   └── WORKSPACE
├── MODULE.bazel
├── WORKSPACE
└── WORKSPACE.bzlmod

.bazelversion

7.2.0

BUILD.bazel

genrule(
    name = "dummy",
    outs = ["dummy.txt"],
    cmd = "touch $@",
)

foo/foo.bzl

def _foo_aspect_impl(target, ctx):
    return []

foo_aspect = aspect(
    implementation = _foo_aspect_impl,
)

foo/WORKSPACE

workspace(name = "foo")

MODULE.bazel

module(name = "my_workspace")

WORKSPACE

workspace(name = "my_workspace")

$ bazel build \
  --enable_bzlmod \
  --noenable_workspace \
  --lockfile_mode=off \
  --aspects=@@foo//:foo.bzl%foo_aspect \
  --override_repository=foo=%workspace%/foo \
  -- //...
Starting local Bazel server and connecting to it...
Analyzing: target //:dummy (2 packages loaded, 0 targets configured)
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.IllegalStateException: Evaluation of SkyKey failed and no dependencies were requested: KeyForBuild{label=@@foo//:foo.bzl, isBuildPrelude=false} IncrementalInMemoryNodeEntry{key=KeyForBuild{label=@@foo//:foo.bzl, isBuildPrelude=false}, value=null, dirtyBuildingState=InitialBuildingState{dirtyState=REBUILDING, signaledDeps=5, externalDeps=0, dirtyDirectDepIndex=0}, version=com.google.devtools.build.skyframe.Version$MinimalVersion@5ba14436, directDeps=GroupedDeps{size=5, elements=[Starlark @_builtins, CONTAINING_PACKAGE_LOOKUP:@@foo//, FILE:[<output_base>/external/foo]/[foo.bzl], PRECOMPUTED:starlark_semantics, Key{repoName=@@foo, rootModuleShouldSeeWorkspaceRepos=true}]}, reverseDeps=ReverseDeps{reverseDeps=[], singleReverseDep=false, dataToConsolidate=[BuildTopLevelAspectsDetailsKey{topLevelAspectsClasses=[@@foo//:foo.bzl%foo_aspect], topLevelAspectsParameters={}}]}}
    at com.google.common.base.Preconditions.checkState(Preconditions.java:835)
    at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:690)
    at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:414)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

Which operating system are you running Bazel on?

Linux / amd64

What is the output of `bazel info release`?

release 7.2.0

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Wyverald commented 3 weeks ago

Thank you for the crystal clear repro instructions! Looking now.

Wyverald commented 3 weeks ago

The crux of the issue is that we have no way to figure out the repo mapping of @@foo, as its definition doesn't exist anywhere (other than the --override_repository flag, that is). With --enable_workspace, we happen to have a fallback in the WORKSPACE file (which gives an empty repo mapping for nonexistent repos); this fallback is not there with --noenable_workspace.

Now, it's not completely clear to me what the correct behavior should be. What should we use as the repo mapping of @@foo? If we give it an empty mapping, it essentially means that @@foo cannot use any labels with a repo part (e.g. load("@bazel_skylib//... would fail). As an alternative, it's arguable that we should just outright fail, since @@foo isn't defined anywhere. @lbcjbb, could you share a bit about your use case with the "external aspect"? Is it for IDE usage?

In any case, we definitely shouldn't crash. So this is still a legit bug.

lbcjbb commented 3 weeks ago

I have no direct use for it. However, the Bazel plugin for Intellij yes. This plugin needs to execute a command similar to the one below:

bazel build \
  --tool_tag=ijwb:IDEA:ultimate \
  --keep_going \
  --build_event_binary_file=<tmpfile> \
  --nobuild_event_binary_file_path_conversion \
  --curses=no \
  --color=yes \
  --progress_in_terminal_title=no \
  --noexperimental_run_validations \
  --aspects=@@intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect \
  --override_repository=intellij_aspect=$HOME/.local/share/JetBrains/IntelliJIdea2024.1/ijwb/aspect \
  --output_groups=intellij-resolve-go,intellij-resolve-java,intellij-resolve-py,intellij-info-generic,intellij-info-go,intellij-info-java,intellij-info-py \
  -- \
  //...

I mitigated the problem by configuring the option --enable_workspace option in the build_flags to override the --noenable_workspace option of my .bazelrc file.

fmeum commented 3 weeks ago

Looking at the aspect source, I think that IntelliJ should be using --override_module instead with Bzlmod. With Bazel 8, it would need to add a dependency on rules_java to access Java providers, even if the main module may not have such a dependency. Any synthetic repo mapping we could produce wouldn't work in this situation. But yes, it shouldn't crash.

Wyverald commented 3 weeks ago

Problem is that --override_module doesn't add a dep. So we'd really need an --add_module or some such... which would then mess with version resolution.

We did discuss this with the JetBrains people before, especially as they might want to support e.g. providers in rules_kotlin without having to force the user into fetching rules_kotlin. I can't remember what conclusion we reached (if any).

cc @agluszak

bazelbuild / bazel