bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.06k stars 4.04k forks source link

Intermittent `java.lang.IllegalStateException: closed` from `bazel info` #22149

Closed joeljeske closed 2 months ago

joeljeske commented 5 months ago

Description of the bug:

Occasionally, I receive the following error stack when I run bazel info peak-heap-size used-heap-size max-heap-size used-heap-size-after-gc.

Bazel still returns the proper values for those info properties, but it also prints this stack to stderr ~5% of the time. I do have --remote_executor set.

(08:54:10) ERROR: java.lang.IllegalStateException: closed
--
  | at com.google.devtools.build.lib.remote.grpc.SharedConnectionFactory.close(SharedConnectionFactory.java:80)
  | at com.google.devtools.build.lib.remote.grpc.DynamicConnectionPool.close(DynamicConnectionPool.java:64)
  | at com.google.devtools.build.lib.remote.ReferenceCountedChannel$1.deallocate(ReferenceCountedChannel.java:54)
  | at io.netty.util.AbstractReferenceCounted.handleRelease(AbstractReferenceCounted.java:86)
  | at io.netty.util.AbstractReferenceCounted.release(AbstractReferenceCounted.java:76)
  | at com.google.devtools.build.lib.remote.ReferenceCountedChannel.release(ReferenceCountedChannel.java:201)
  | at com.google.devtools.build.lib.remote.GrpcRemoteExecutor.close(GrpcRemoteExecutor.java:250)
  | at com.google.devtools.build.lib.remote.RemoteActionContextProvider.afterCommand(RemoteActionContextProvider.java:246)
  | at com.google.devtools.build.lib.remote.RemoteModule.afterCommandTask(RemoteModule.java:864)
  | at com.google.devtools.build.lib.remote.RemoteModule.lambda$afterCommand$4(RemoteModule.java:842)
  | at com.google.devtools.build.lib.runtime.BlockWaitingModule.lambda$submit$0(BlockWaitingModule.java:73)
  | at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
  | at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
  | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  | at java.base/java.lang.Thread.run(Unknown Source)

Which category does this issue belong to?

Remote Execution

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

release 7.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

iancha1992 commented 5 months ago

@joeljeske Could you please provide sample code to reproduce this issue? Thanks

tjgq commented 2 months ago

I ran bazel info peak-heap-size used-heap-size max-heap-size used-heap-size-after-gc in a loop with --remote_executor set, but I couldn't reproduce it. There's probably some other flag that need to be set? Please reopen with a self-contained repro.