Closed aherrmann closed 2 years ago
It works locally, but on CI this seems to break Bazel
CANCELLED: Thread interrupted
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ActionLookupData{actionLookupKey=ConfiguredTargetKey{label=//tests/sh_binaries:custom_rule, config=BuildConfigurationValue.Key[96d6638d77e294d5ea45abfa4bdf12d435d0d7fb38e15d839ac9c6ef83541e29]}, actionIndex=0}' (requested by nodes 'ArtifactNestedSetKey{rawChildren=[File:[[<execution_root>]bazel-out/k8-fastbuild/bin]tests/sh_binaries/custom_rule_run_output, File:[[<execution_root>]bazel-out/k8-fastbuild/bin]tests/sh_binaries/custom_rule_run_shell_output]}')
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:563)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:398)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalStateException: File:[/home/runner/.cache/bazel/_bazel_runner/994b3757b3bfe7aba5edbda5c2c312aa/external/bazel_tools[source]]tools/bash/runfiles/runfiles.bash is not present in declared outputs: [File:[[<execution_root>]bazel-out/k8-fastbuild/bin]tests/sh_binaries/custom_rule_run_output]
at com.google.common.base.Preconditions.checkState(Preconditions.java:824)
at com.google.devtools.build.lib.skyframe.ActionMetadataHandler.getMetadata(ActionMetadataHandler.java:247)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$DelegatingPairFileCache.getMetadata(SkyframeActionExecutor.java:1733)
at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.lambda$buildFromActionInputs$1(DirectoryTreeBuilder.java:137)
at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.build(DirectoryTreeBuilder.java:201)
at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.buildFromActionInputs(DirectoryTreeBuilder.java:123)
at com.google.devtools.build.lib.remote.merkletree.DirectoryTreeBuilder.fromActionInputs(DirectoryTreeBuilder.java:62)
at com.google.devtools.build.lib.remote.merkletree.MerkleTree.build(MerkleTree.java:140)
at com.google.devtools.build.lib.remote.RemoteExecutionService.buildRemoteAction(RemoteExecutionService.java:242)
at com.google.devtools.build.lib.remote.RemoteSpawnCache.lookup(RemoteSpawnCache.java:97)
at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:139)
at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:106)
at com.google.devtools.build.lib.actions.SpawnStrategy.beginExecution(SpawnStrategy.java:47)
at com.google.devtools.build.lib.exec.SpawnStrategyResolver.beginExecution(SpawnStrategyResolver.java:65)
at com.google.devtools.build.lib.analysis.actions.SpawnAction.beginExecution(SpawnAction.java:331)
at com.google.devtools.build.lib.actions.Action.execute(Action.java:127)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$5.execute(SkyframeActionExecutor.java:855)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.continueAction(SkyframeActionExecutor.java:1016)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.run(SkyframeActionExecutor.java:975)
at com.google.devtools.build.lib.skyframe.ActionExecutionState.runStateMachine(ActionExecutionState.java:129)
at com.google.devtools.build.lib.skyframe.ActionExecutionState.getResultOrDependOnFuture(ActionExecutionState.java:81)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:472)
at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:834)
at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:307)
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:477)
... 4 more
I can reproduce the issue locally by enabling the remote cache
The issue was caused by passing only the input_manifests
portion of a ctx.resolve_tools
to ctx.actions.run(_shell)
and skipping the inputs
part. It works fine locally, but crashes Bazel when remote caching is enabled.
I was able to create a minimal repro for that Bazel crash and raised an issue upstream.
@avdv Thanks! Fixed.
This rule can bundle multiple executable files tracked by Bazel into a single target that can be used in genrules or custom rules to access those tools, either through make variables in genrules, or through the ShBinariesInfo provider in custom rules.
This is a part of https://github.com/tweag/rules_sh/issues/19
Note, this PR includes docstrings on the new rule, but this repository does not currenly have Stardoc documentation generation configured. Instead of manually updating the README, this PR defers the README update to a time when Stardoc documentation generation is configured.