Command '['bazel', 'dump', '--action_cache']' returned non-zero exit status 9

[Kayaking-zh]

I use bazel 6.1.1 with bzlmod, when I execute bazel run :refresh_compile_commands, it fails with following traceback but no error message. I don't know why and what could I do.

Traceback (most recent call last):
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.check_python_version.py", line 15, in <module>
    refresh_compile_commands.main()
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 1416, in main
    compile_command_entries.extend(_get_commands(target, flags))
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 1278, in _get_commands
    yield from _convert_compile_commands(parsed_aquery_output)
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 1158, in _convert_compile_commands
    for source_files, header_files, compile_command_args in outputs:
  File "/home/project/bazel-cache/external/rules_python~0.25.0~python~python_3_11_x86_64-unknown-linux-gnu/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/external/rules_python~0.25.0~python~python_3_11_x86_64-unknown-linux-gnu/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/external/rules_python~0.25.0~python~python_3_11_x86_64-unknown-linux-gnu/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/external/rules_python~0.25.0~python~python_3_11_x86_64-unknown-linux-gnu/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/project/bazel-cache/external/rules_python~0.25.0~python~python_3_11_x86_64-unknown-linux-gnu/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 1122, in _get_cpp_command_for_files
    source_files, header_files = _get_files(compile_action)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 679, in _get_files
    header_files = _get_headers(compile_action, source_file)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 597, in _get_headers
    headers, should_cache = _get_headers_gcc(compile_action, source_path, compile_action.actionKey)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 243, in _get_headers_gcc
    if action_key in _get_bazel_cached_action_keys():  # Safe because Bazel only holds one cached action key per path, and the key contains the path.
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/execroot/_main/bazel-out/k8-fastbuild/bin/refresh_compile_commands.runfiles/_main/refresh_compile_commands.py", line 132, in _get_bazel_cached_action_keys
    action_cache_process = subprocess.run(
                           ^^^^^^^^^^^^^^^
  File "/home/project/bazel-cache/external/rules_python~0.25.0~python~python_3_11_x86_64-unknown-linux-gnu/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['bazel', 'dump', '--action_cache']' returned non-zero exit status 9.

If I run bazel dump --action_cache directly, it succeeds. I use git 2.45.2, gcc 8.3.1 and bazel-compile-commands-extractor a14ad3a64e7bf398ab48105aaa0348e032ac87f8

[cpsauer]

Uh oh. Any chance it works with a newer version of Bazel? (haven't seen this one before and am worried it might be an underlying bazel issue, since that shouldn't fail)

To get more info, any chance you'd be down to just quickly monkey patch refresh_compile_commands.py to print out some more info? (should be under external/. Just commend out the capture of stderr=subprocess.PIPE)

Thanks for reporting--and sorry about the rough edge.

Chris

[Kayaking-zh]

I tried bazel 7.1.1 and it still didn't work. Then I comment out stderr=subprocess.PIPE and got lots of error message as following.

FATAL: Exiting because the lock is held and --noblock_for_lock was given.
Another command holds the client lock: 
pid=27413
owner=client
cwd=/home/project
tty=/dev/pts/8

It's because I have --noblock_for_lock in .bazelrc, after i delete it, all works.