Another pybind11/hermetic python version problem

[jmbalasalle]

Hello again! I'm having trouble with python versions.

I'm seeing the same issue that was reported here: https://github.com/hedronvision/bazel-compile-commands-extractor/pull/139

When I run bazel run @hedron_compile_commands//:refresh_all I see the following:

user@mst-develop-aa3fa745-5f46-4332-8311-92a2ab6f8445 /scratch/user/Mst $ bazel run @hedron_compile_commands//:refresh_all
INFO: Build options --features and --host_features have changed, discarding analysis cache.
INFO: Analyzed target @hedron_compile_commands//:refresh_all (0 packages loaded, 2664 targets configured).
INFO: Found 1 target...
Target @hedron_compile_commands//:refresh_all up-to-date:
  bazel-bin/external/hedron_compile_commands/refresh_all.py
  bazel-bin/external/hedron_compile_commands/_refresh_all
INFO: Elapsed time: 0.424s, Critical Path: 0.04s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/external/hedron_compile_commands/refresh_all
>>> Analyzing commands used in @//...
>>> While locating the headers you use, we encountered a compiler warning or error.
    No need to worry; your code doesn't have to compile for this tool to work.
    However, we'll still print the errors and warnings in case they're helpful for you in fixing them.
    If the errors are about missing files that Bazel should generate:
        You might want to run a build of your code with --keep_going.
        That way, everything possible is generated, browsable and indexed for autocomplete.
    But, if you have *already* built your code successfully:
        Please make sure you're supplying this tool with the same flags you use to build.
        You can either use a refresh_compile_commands rule or the special -- syntax. Please see the README.
        [Supplying flags normally won't work. That just causes this tool to be built with those flags.]
    Continuing gracefully...
In file included from external/pybind11/include/pybind11/detail/../attr.h:13,
                 from external/pybind11/include/pybind11/detail/class.h:12,
                 from external/pybind11/include/pybind11/pybind11.h:13,
                 from src/pyBindings.cpp:5:
external/pybind11/include/pybind11/detail/../detail/common.h:269:6: error: #error "PYTHON < 3.6 IS UNSUPPORTED. pybind11 v2.9 was the last to support Python 2 and 3.5."
  269 | #    error "PYTHON < 3.6 IS UNSUPPORTED. pybind11 v2.9 was the last to support Python 2 and 3.5."
      |      ^~~~~
>>> Finished extracting commands for @//...

I cannot, for the life of me, figuring out how it's picking up this ancient version of python. I don't even have a version that old installed. I'm using Ubuntu 20.04.

I'm using the latest commit, 199ca857b05a7a4dbb332e8d229158feb3f82638. Here is the relvant section of my WORKSPACE file:

# Hedron bazel compile_commands.json generator
# currently, this needs to go before the rules_pkg
http_archive(
    name = "hedron_compile_commands",
    url = "https://github.com/hedronvision/bazel-compile-commands-extractor/archive/199ca857b05a7a4dbb332e8d229158feb3f82638.tar.gz",
    strip_prefix = "bazel-compile-commands-extractor-199ca857b05a7a4dbb332e8d229158feb3f82638",
)

load("@hedron_compile_commands//:workspace_setup.bzl", "hedron_compile_commands_setup")
hedron_compile_commands_setup()
load("@hedron_compile_commands//:workspace_setup_transitive.bzl", "hedron_compile_commands_setup_transitive")
hedron_compile_commands_setup_transitive()
load("@hedron_compile_commands//:workspace_setup_transitive_transitive.bzl", "hedron_compile_commands_setup_transitive_transitive")
hedron_compile_commands_setup_transitive_transitive()
load("@hedron_compile_commands//:workspace_setup_transitive_transitive_transitive.bzl", "hedron_compile_commands_setup_transitive_transitive_transitive")
hedron_compile_commands_setup_transitive_transitive_transitive()

http_archive(
    name = "rules_pkg",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/0.9.1/rules_pkg-0.9.1.tar.gz",
        "https://github.com/bazelbuild/rules_pkg/releases/download/0.9.1/rules_pkg-0.9.1.tar.gz",
    ],
    sha256 = "8f9ee2dc10c1ae514ee599a8b42ed99fa262b757058f65ad3c384289ff70c4b8",
)
load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
rules_pkg_dependencies()

# Update the SHA and VERSION to the lastest version available here:
# https://github.com/bazelbuild/rules_python/releases.

SHA="d70cd72a7a4880f0000a6346253414825c19cdd40a28289bdf67b8e6480edff8"
http_archive(
   name = "rules_python",
   sha256 = SHA,
   strip_prefix = "rules_python-0.28.0",
   url = "https://github.com/bazelbuild/rules_python/releases/download/0.28.0/rules_python-0.28.0.tar.gz",
)
load("@rules_python//python:repositories.bzl", "py_repositories")
py_repositories()

git_repository(
   name = "pybind11_bazel",
   commit = "2e5f4a6beece3b92d2f87744f305eb52b6852aa9",
   remote = "https://github.com/pybind/pybind11_bazel.git",
)

new_git_repository(
   name = "pybind11",
   build_file = "@pybind11_bazel//:pybind11.BUILD",
   commit = "8a099e44b3d5f85b20f05828d919d2332a8de841",
   remote = "https://github.com/pybind/pybind11.git",
)

load("@pybind11_bazel//:python_configure.bzl", "python_configure")
python_configure(name = "local_config_python", python_version = "3")

The bazel target in question builds just fine from the command line.

I still get the resulting compile_commands.json file and it seems fine. So this is not really an issue, but I'm curious what you think.

[cpsauer]

Hey James :)

It must be that it's inferring system python, right? (We aren't actually using system python for this tool anymore, but rather hermetic 3.11, as per that PR you linked.)

I'm not totally sure, but I'm going to guess that maybe pybind does an internal transition to configure its python version, if not using the system one. If you haven't already, would you be down to try using a refresh_compile_commands rule to point this tool to you your top-level target (this section of the readme) and see if that works? That'd cause it to pick up any configuration transitions applied in your normal build.

Cheers, Chris

[cpsauer]

FYI: Reverted rules_python in https://github.com/hedronvision/bazel-compile-commands-extractor/commit/0b821b7e4286aec887757461366f6eaaa0972cb9 because of lots of issues :( Tracking restoration in https://github.com/hedronvision/bazel-compile-commands-extractor/issues/168.

[jmbalasalle]

After all the recent updates to this tool, as well as cleaning up my own WORKSPACE file, I am no longer having this issue. It was likely operator error anyway.