Unable to build Python gRPC service with Bazel

[q0dr]

What version of gRPC and what language are you using?

gRPC 1.63.1 python 3.12.1

What operating system (Linux, Windows,...) and version?

macOS Sonoma 14.5

What runtime / compiler are you using (e.g. python version or version of gcc)

python 3.12.1

rgenter@RGENTER-M-040N ~ % gcc --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
rgenter@RGENTER-M-040N ~ % 

What did you do?

I have a test that demonstrates that with Bazel 7.2.0, protobuf 27.1 and gRPC 1.63.1, a gRPC service written in Python fails to build on a MacBook Pro with an Apple M3 Pro chip.

My MODULE.bazel file:

"""Test demonstrating failure between Bazel, gRPC and Python."""
module(name = "test", version = "1.0")

bazel_dep(name = "grpc", version = "1.63.1", repo_name = "com_github_grpc_grpc")
bazel_dep(name = "protobuf", version = "27.1")

use_repo_rule("@@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
use_repo_rule("@@com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")
use_repo_rule("@@com_github_grpc_grpc//baze::grpc_python_deps.bzl", "grpc_python_deps")

My BUILD.bazel file:

package(default_visibility = ["//visibility:public"])

load("@com_github_grpc_grpc//bazel:cc_grpc_library.bzl", "cc_grpc_library")
load("@com_github_grpc_grpc//bazel:python_rules.bzl", "py_grpc_library", "py_proto_library")

proto_library(
    name = "service_proto",
    srcs = ["service.proto"],
)

cc_proto_library(
    name = "service_cc_proto",
    deps = [":service_proto"],
)

cc_grpc_library(
    name = "service_cc_grpc",
    srcs = [":service_proto"],
    grpc_only = True,
    deps = [":service_cc_proto"],
)

py_proto_library(
    name = "service_py_proto",
    deps = [":service_proto"],
)

py_grpc_library(
    name = "service_py_grpc",
    srcs = [":service_proto"],
    deps = [":service_py_proto"],
)

My service.proto file:

syntax = "proto3";

package test;

service MyService {
    rpc MyMethod(MyRequest) returns (MyResponse);
}

message MyRequest {
    optional string request = 1;
}

message MyResponse {
    optional string response = 1;
}

When I do:

$ bazel clean --expunge
$ bazel build //:service_py_grpc

I get:

ERROR: /private/var/tmp/_bazel_rgenter/a1a8280059cd89e552f80330d54f1415/external/grpc~/src/python/grpcio/grpc/_cython/BUILD.bazel:25:12: Linking external/grpc~/src/python/grpcio/grpc/_cython/cygrpc.so failed: (Exit 1): cc_wrapper.sh failed: error executing CppLink command (from target @@grpc~//src/python/grpcio/grpc/_cython:cygrpc.so) external/bazel_tools~cc_configure_extension~local_config_cc/cc_wrapper.sh @bazel-out/darwin_arm64-fastbuild/bin/external/grpc~/src/python/grpcio/grpc/_cython/cygrpc.so-2.params

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Undefined symbols for architecture arm64:
  "_PyArg_UnpackTuple", referenced from:
      __Pyx_Coroutine_Throw(_object*, _object*) in cygrpc.o
      __Pyx_async_gen_athrow_send(__pyx_PyAsyncGenAThrow*, _object*) in cygrpc.o
  "_PyAsyncGen_Type", referenced from:
      __Pyx_PyGen_Send(PyGenObject*, _object*) in cygrpc.o
  "_PyBaseObject_Type", referenced from:
      __Pyx_InBases(_typeobject*, _typeobject*) in cygrpc.o
      __Pyx_setup_reduce(_object*) in cygrpc.o
      __pyx_tp_new_7_cython_6cygrpc__ChannelArg(_typeobject*, _object*, _object*) in cygrpc.o
      __pyx_tp_new_7_cython_6cygrpc__ChannelArgs(_typeobject*, _object*, _object*) in cygrpc.o
      __pyx_tp_new_7_cython_6cygrpc_Call(_typeobject*, _object*, _object*) in cygrpc.o
      __pyx_tp_new_7_cython_6cygrpc__CallState(_typeobject*, _object*, _object*) in cygrpc.o
      __pyx_tp_new_7_cython_6cygrpc__ChannelState(_typeobject*, _object*, _object*) in cygrpc.o
      ...

followed by several thousand more lines complaining about undefined symbols, all of which start with _Py or __Py. This seems to come from Cython, which gRPC apparently depends upon for Python.

If I try to build a C++ service using:

$ bazel clean --expunge
$ bazel build //:service_cc_grpc

The build succeeds.

What did you expect to see?

A successful build.

What did you see instead?

See above.

Anything else we should know about your project / environment?

Nothing that I can think of. I'm happy to answer questions.

NOTE: This is also filed as https://github.com/grpc/grpc.io/issues/1307

[gmweaver]

Reproduced issue above with Bazel 7.1.2, note that this seems to be Mac specific (maybe arm64 specific) as all our py targets work fine on linux builds.

[jgertm]

Running into the same issue with Bazel 6.5.0 on a M1 Mac with Sonoma 14.6

[sourabhsinghs]

@q0dr We don't claim support of MODULE.bazel . We'll welcome any contribution to add support for this.

[mbland]

I was able to reproduce this in a gist, Reproduction of grpc/grpc#37251, and to fix it in https://github.com/bazelbuild/bazel-central-registry/pull/2685. Though by the looks of the presubmit checks, this may necessitate releasing an updated module version instead.

[mbland]

https://github.com/bazelbuild/bazel-central-registry/pull/2685 is in. @q0dr can you update your grpc module to version 1.66.0.bcr.1 and see if it fixes your build?