Embedding Python produces "dyld[69334]: symbol not found in flat namespace '_PyBaseObject_Type'"

[vymao]

I'm trying to embed Python into a C++ application via Bazel. My WORKSPACE includes the following:

    http_archive(
        name = "pybind11_bazel",
        strip_prefix = "pybind11_bazel-master",
        urls = ["https://github.com/pybind/pybind11_bazel/archive/master.zip"],
    )
            # We still require the pybind library.
    http_archive(
        name = "pybind11",
        build_file = "@pybind11_bazel//:pybind11.BUILD",
        strip_prefix = "pybind11-2.11.1",
        urls = ["https://github.com/pybind/pybind11/archive/v2.11.1.tar.gz"],
    )
load("@pybind11_bazel//:python_configure.bzl", "python_configure")
python_configure(name = "local_config_python")

and my BUILD looks like

cc_binary(
    name = "main",
    srcs = ["main.cc"],
    copts = ["-Ilib/include"],
    deps = [
        ... # extra dependencies
        "@pybind11//:pybind11_embed",
        "@pybind11",
    ]
)

and my main.cc file uses #include <pybind11/embed.h> and namespace py = pybind11;. But when I try and do py::scoped_interpreter guard{};, I get dyld[69334]: symbol not found in flat namespace '_PyBaseObject_Type'. Not sure why this is? It compiles without issue. Could this be because I use Anaconda environments to manage Python dependencies?

[rwgk]

I cannot help a lot, but

• almost certainly: this symbol is provided by Python (e.g. libpython3.12.a), that needs to be linked in.
• I don't know to specify that in the deps. I hope this clue gets you on the right track.

[vymao]

In the repo description, there is the following:

@pybind11//:pybind11_embed currently supports Python 3 MacOS/Ubuntu/Debian environments:

pyenv pipenv virtualenv

If pybind11_embed doesn't work with your embedded Python project, add @pybind11 as a dependency to your cc_binary and follow the instructions for manually retrieving the build flags.

Considering I use conda, could this be the issue? I'm not 100% sure how to resolve this using Conda though.

[rwgk]

Considering I use conda, could this be the issue? I'm not 100% sure how to resolve this using Conda though.

Sorry, no idea.

I'd try to add to copts, -L and -lpthon3.12 or similar (maybe with -Wl,). The main problem is to pint-point the correct Python runtime library.

[vymao]

Thanks. But I wonder: considering Bazel is able to compile/build without issue, is it strictly a linking issue? I think copts will be for compilation, no?

[rwgk]

We don't test embedding, at all, internally or externally (AFAIK, almost certain). I hope you can figure it out with the clues I can offer, or someone else can help.

[junyer]

As of the v2.11.1.bzl.2 release, pybind11_bazel uses rules_python and the pybind11_embed target no longer exists. Please try using @rules_python//python/cc:current_py_cc_libs instead; if that has any problems, please file a new issue (feel free to loop me in!) with the https://github.com/bazelbuild/rules_python project.