Unable to run grpcio wheel on alpine arvm7

allenporter commented 3 years ago

What version of gRPC and what language are you using?

1.41.0 in python. Any grpcio release over 1.31.0 exhibits a variation on the problem.

What operating system (Linux, Windows,...) and version?

Linux arm32v7/alpine:3.12

What runtime / compiler are you using (e.g. python version or version of gcc)

Python 3.8.10 though other versions also show the same problem

What did you do?

Given an example Dockerfile like this:

FROM arm32v7/alpine:3.12

RUN apk add --no-cache python3 py3-pip
RUN apk add libstdc++ libc6-compat
RUN python3 --version

WORKDIR /code
COPY requirements.txt .
COPY example.py .

RUN pip3 install --no-cache-dir --only-binary=grpcio -r requirements.txt

CMD python3 ./example.py

Note that this also has problems finding the shared library without libstdc++ and libc6-compat but the errors has changed across grpc versions.

requirements.txt:

grpcio==1.41.0
google-cloud-pubsub

example.py

#!/usr/bin/python

from google.cloud import pubsub_v1

def main():
  print("Hello world")

if __name__ == "__main__":
  main()

Then run a build like this:

$ docker build --tag grpcio-armv7 .

What did you expect to see?

$ docker run grpcio-armv7
Hello world

Note, that this is what I see for alpine images on other architectures or different armv7 images like this:

FROM arm32v7/python:3.8-buster

What did you see instead?

$ docker run grpcio-armv7
Traceback (most recent call last):                                              
  File "./example.py", line 3, in <module>                                      
    from google.cloud import pubsub_v1                                          
  File "/usr/lib/python3.8/site-packages/google/cloud/pubsub_v1/__init__.py", li
ne 17, in <module>                                                              
    from google.cloud.pubsub_v1 import types                                    
  File "/usr/lib/python3.8/site-packages/google/cloud/pubsub_v1/types.py", line 
25, in <module>                                                                 
    from google.api_core import gapic_v1                                        
  File "/usr/lib/python3.8/site-packages/google/api_core/gapic_v1/__init__.py", 
line 16, in <module>                                                            
    from google.api_core.gapic_v1 import config                                 
  File "/usr/lib/python3.8/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/usr/lib/python3.8/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/usr/lib/python3.8/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: Error relocating /usr/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-arm-linux-gnueabihf.so: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEv: symbol not found

When using an older version of grpcio such as 1.38.X, I see an error message like:

Traceback (most recent call last):
  File "./example.py", line 3, in <module>
    from google.cloud import pubsub_v1
  File "/usr/lib/python3.8/site-packages/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
    from google.cloud.pubsub_v1 import types
  File "/usr/lib/python3.8/site-packages/google/cloud/pubsub_v1/types.py", line 25, in <module>
    from google.api_core import gapic_v1
  File "/usr/lib/python3.8/site-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
    from google.api_core.gapic_v1 import config
  File "/usr/lib/python3.8/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/usr/lib/python3.8/site-packages/grpc/__init__.py", line 23, in <module>
    from grpc._cython import cygrpc as _cygrpc
ImportError: Error relocating /usr/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-arm-linux-gnueabihf.so: __strftime_l: symbol not found

This is similar to the first post of https://github.com/grpc/grpc/issues/24808 where the armv7 binary wheel has problems with alpine.

Anything else we should know about your project / environment?

This seems to have been an armv7 plus alpine issue that first happened last year. I have some history in https://github.com/home-assistant/core/issues/56669 though this was first seen with https://github.com/home-assistant/core/issues/40266 and https://github.com/home-assistant/core/issues/40148

Also note above that the Dockerfile explicitly adds libstdc++ libc6-compat which are not necessary for other platforms besides arvm7. Ideally that should not be needed and this could run on musl directly.

The discussion in https://github.com/grpc/grpc/issues/26620 seems quite relevant. Is building musllinux wheels for armv7 the right solution?

allenporter commented 3 years ago

@lidizheng curious if you had additional thoughts on musllinux wheels as a potential answer to this and when you might realistically be able to consider this?

To elaborate on my motivation: grpc on alpine arvm7 would help get a lot of projects off of old libraries. The python version pins restricting dependencies to only old packages are getting difficult to manage, so it would be really helpful to have a path forward to avoid having to maintain that.

lidizheng commented 3 years ago

The alpine support is on the road map. But alpine plus ARM, which might require actual ARM machines or complex cross compiling.

There is a similar minimum Docker image you may find helpful, and it's glibc based (so it can use manylinux wheels): https://github.com/phusion/baseimage-docker. After all, if we compile on alpine, then the size of dependencies might defeat the purpose of using alpine.

Both compilation errors come from the C++ Abseil library, I'm not sure how to fix. Can you try again with up-to-date C++ compiler?

allenporter commented 3 years ago

Great, glad to hear this is planned.

The target is a popular open source project that runs in python (home assistant) which i'm almost positive won't switch base docker images because of grpc. They're preference would be for grpc to continue to work again on alpine like it in grpcio==1.31.0.

Can you help me understand your last comment a bit better: My impression is these are link errors from installing binary packages. How do compiler errors and the C++ compiler come into play?

Do you have a rough idea for the scope of effort involved here? this is definitely not my expertise, but i'm very motivated to get this unstuck given the huge amount of pain this has been causing and bad will in the community, if you gave me some pointers or rough scoping.

lidizheng commented 3 years ago

For fixing the compilation, to be honest, I will need machine and repro to dig deeper.

On the other hand, it would be great if you can help to investigate this issue. According to the missing symbol, it looks like the libc++ itself is causing problem. Probably you could try to use GRPC_PYTHON_BUILD_WITH_STATIC_LIBSTDCXX=1 to bundle libc++ into the artifact. (This switch is created by @jtattermusch to fix another issue with ARM builds https://github.com/grpc/grpc/blob/master/setup.py#L165)

If this doesn't work, there are many other nobs that can be used to tune compilation: https://github.com/grpc/grpc/blob/master/setup.py#L117

For a short term fix, I think we can figure out a set of flags we need to set on Alpine+ARM and add them into our setup.py.

(If stuck, feel free to post more build logs here.)

allenporter commented 3 years ago

OK, i can give that a try. I have a raspberry pi 4 with a 32-bit os installed on it to get armv7 that I set up to repro this problem on behalf of users. Compilation is just unbearably slow, so I stuck to just repo with wheels. I can give this a try, or maybe i can figure out how to get a cross compile going.

I assume grpc is cross compiling wheels when releasing them. Is that a process I can follow? (test on my repro box, adjust flags, repeat) The code base is a bit large, so if you give me a pointer that would save me a little time.

lidizheng commented 3 years ago

I wonder if you tried to install wheels from https://www.piwheels.org/project/grpcio/ ? They have community maintained releases of grpcio for Raspberry Pi users.

Sorry, I'm not familiar with the cross compiling part of automation. @jtattermusch if you have second, can you help to share a few pointers?

allenporter commented 3 years ago

When I use the piwheels images I see #26094 with both 1.31.0 and 1.41.0.

From build:

Collecting grpcio==1.41.0
  Downloading https://www.piwheels.org/simple/grpcio/grpcio-1.41.0-cp39-cp39-linux_armv7l.whl (45.2 MB)

From the run:

  File "/usr/lib/python3.9/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: cannot import name 'cygrpc' from 'grpc._cython' (/usr/lib/python3.9/site-packages/grpc/_cython/__init__.py)

I'll start poling around with compiler flags without a wheel in the mean time.

k-s-t-i commented 3 years ago

Issue also happening running through Bazel on Apple M1.

allenporter commented 3 years ago

@jtattermusch I could use a couple pointers if you have a moment.

I am trying to set up a development environment to cross compile armv7 python wheels on a faster machine, then i can try them out on an armv7 machine, adjust compiler flags, and repeat. I see a series of python scripts and docker containers related to building (e.g. what was touched / changed in PR #26279) but am having a little trouble quickly connecting the dots on how these are all typically run. Have any pointers on how the build system works?

allenporter commented 2 years ago

Now since 1.31.0 was not built for python 3.9 it means users are forced to build gprcio from source, so this is getting more dire are folks are dropping python 3.8 support. This affects more than armv7 anyone that pins grpcio to an old version likely does this for all platforms, not just one.

lidizheng commented 2 years ago

I have manually generated gRPC Python wheels for musllinux Aarch64 for Python 3.9 and 3.10. Here are the links:

allenporter commented 2 years ago

We chatted about how the target needed is really arvm7.

I did some testing and determined building a wheel from source on armv7 works fine:

Building from pip3 install grpcio --no-binary ":all:" to build from source works fine on armv7
Building from grpc source project via python3 ./setup.py bdist_wheel works fine on armv7

I think a next target is to try to manually reproduce building the wheel that is built for armv7.

When running on an intel machine, I was able to use python3 tools/run_tests/task_runner.py -b XXX to build a wheel for x64. That works fine.

Then trying to run python3 tools/run_tests/task_runner.py -b python_linux_extra_armv7_cp39-cp39 on x64 fails with a LONG_BIT mismatch error. (log report_taskrunner_sponge.xml.log)

I may now try this on an armv7 machine.

allenporter commented 2 years ago

Running the task running on an armv7 machine does not work as its expecting to run a cross compile for an x64 machine:

$ python3 tools/run_tests/task_runner.py -b python_linux_extra_armv7_cp39-cp39

...
Status: Downloaded newer image for dockcross/linux-armv7:latest
 ---&gt; e914a6d02596
Step 2/11 : RUN apt update &amp;&amp; apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev                                  libnss3-dev libssl-dev libreadline-dev libffi-dev &amp;&amp; apt-get clean
 ---&gt; [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
 ---&gt; Running in d7750d69746a

This makes sense given https://github.com/grpc/grpc/blob/master/tools/dockerfile/grpc_artifact_python_linux_armv7/Dockerfile says The dockcross/linux-armv7 image is a x86_64 image with crosscompilation toolchain installed so I think getting that to work on x86_64 is my best path forward.

lidizheng commented 2 years ago

I see you have completed compiling an armv7 wheel. Is this sufficient for your debugging need?

allenporter commented 2 years ago

As we talked about over chat, the plan is now to get cross compiling going and see what's wrong since we want to fix the build system that cross compiled the wheels.

I can see now the problem in install_python_for_wheel_crosscompilation.sh is when running the cross compilation the ./configure step is failing. Here is where configure failures to setup the cross complile target:

checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for python3.9... python3.9
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... "linux"
checking for gcc... /usr/xcc/armv7l-linux-musleabihf-cross/bin/armv7l-linux-musleabihf-gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... [91mconfigure: error: in `/work/Python-3.9.2':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

The shell script sets set -ex, however because of the way the the configure line is written it doesn't actually fail:

# When building the crosscompiled native extension, python will automatically add its include directory
# that contains "Python.h" and other headers. But since we are going to be crosscompiling
# the wheels, we need the pyconfig.h that corresponds to the target architecture.
# Since pyconfig.h is the only generated header and python itself (once built) doesn't
# really need this header anymore, we simply generate a new pyconfig.h using our crosscompilation
# toolchain and overwrite the current ("wrong") version in the python's include directory.
./configure && cp pyconfig.h "${PYTHON_PREFIX}"/include/python*

When configure fails, it keeps going anyway.

e.g. given example.sh

#!/bin/bash

set -ex

/bin/false && echo "A"

echo "B"

$ ./example.sh
+ /bin/false
+ echo B
B

allenporter commented 2 years ago

Making progress, I was able to update install_python_for_wheel_crosscompilation.sh and build a cross compiled armv7 wheel for musl linux. I copied a pattern in your other PR for musl linux, with a couple small tweaks.

I had to rename the wheel to something like grpcio-1.43.0.dev0-cp39-cp39-musllinux_1_0_armv7l.whl and was able to install it.

However, I still have some tweaks to make as this is the error I get when trying to use that wheel:

Traceback (most recent call last):
  File "/code/./example.py", line 3, in <module>
    from google.cloud import pubsub_v1
  File "/usr/lib/python3.9/site-packages/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
    from google.cloud.pubsub_v1 import types
  File "/usr/lib/python3.9/site-packages/google/cloud/pubsub_v1/types.py", line 27, in <module>
    from google.api_core import gapic_v1
  File "/usr/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
    from google.api_core.gapic_v1 import config
  File "/usr/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/usr/lib/python3.9/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/usr/lib/python3.9/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: cannot import name 'cygrpc' from 'grpc._cython' (/usr/lib/python3.9/site-packages/grpc/_cython/__init__.py)

I get the same error on both a musl linux and the existing python_linux_extra_armv7_cp39-cp39 build without musl linux, effectively a repro for https://github.com/grpc/grpc/issues/26094

lidizheng commented 2 years ago

I had to rename the wheel to something like grpcio-1.43.0.dev0-cp39-cp39-musllinux_1_0_armv7l.whl and was able to install it.

About the renaming part, PyPI provides a tool to detect the content of the wheel, and rename it to the corresponding policy (like manylinux_2_17, or musllinux_1_1). If there is any error in wheel metadata or in binary, this tool should be able to find out.

pip install -U auditwheel
auditwheel repair *.whl

Cannot import cygrpc means problem with the compiled shared library. If the error persists, I usually unzip the wheel and check the .so file.

allenporter commented 2 years ago

OK -- I had problems finding audithweel for the musl architecture and had to disable it, so that makes sense. However, i'm having the cygrpc problem on both architectures so perhaps i'll focus there first.

This is the same problem in https://github.com/grpc/grpc/issues/26094 so I included a repro case there.

allenporter commented 2 years ago

When looking at the shared libraries on the musllinux arm build:

# readelf -d cygrpc.cpython-39-arm-linux-gnueabihf.so 
Dynamic section at offset 0x91d6000 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000f (RPATH)                      Library rpath: [$ORIGIN/../../grpcio.libs]
 0x00000001 (NEEDED)                     Shared library: [libatomic-42cf962b.so.1.2.0]
 0x00000001 (NEEDED)                     Shared library: [ld-musl-armhf-c0608a5d.so.1]
 0x0000000c (INIT)                       0x92544
 0x0000000d (FINI)                       0x6dc20c
 0x00000019 (INIT_ARRAY)                 0x7ac610
 0x0000001b (INIT_ARRAYSZ)               1168 (bytes)
 0x0000001a (FINI_ARRAY)                 0x7acaa0
 0x0000001c (FINI_ARRAYSZ)               4 (bytes)
 0x00000004 (HASH)                       0x8256d0
 0x6ffffef5 (GNU_HASH)                   0x91dc
 0x00000005 (STRTAB)                     0x86d0f8
 0x00000006 (SYMTAB)                     0x12b80
 0x0000000a (STRSZ)                      251652 (bytes)
 0x0000000b (SYMENT)                     16 (bytes)
 0x00000003 (PLTGOT)                     0x7c3000
 0x00000002 (PLTRELSZ)                   9736 (bytes)
 0x00000014 (PLTREL)                     REL
 0x00000017 (JMPREL)                     0x8ff3c
 0x00000011 (REL)                        0x66d7c
 0x00000012 (RELSZ)                      168384 (bytes)
 0x00000013 (RELENT)                     8 (bytes)
 0x6ffffffe (VERNEED)                    0x66d5c
 0x6fffffff (VERNEEDNUM)                 1
 0x6ffffff0 (VERSYM)                     0x64502
 0x6ffffffa (RELCOUNT)                   17290
 0x00000000 (NULL)                       0x0
    /lib/ld-musl-armhf.so.1 (0xb6edf000)
    libatomic-42cf962b.so.1.2.0 => ./../../grpcio.libs/libatomic-42cf962b.so.1.2.0 (0xb661e000)
    libc.so => /lib/ld-musl-armhf.so.1 (0xb6edf000)
# ldd cygrpc.cpython-39-arm-linux-gnueabihf.so
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyModule_AddObject: symbol not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyInterpreterState_GetID: symbol not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyDict_SetItemString: symbol not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyUnicode_Compare: symbol not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyFrame_New: symbol not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: _PyUnicode_FastCopyCharacters: symbol not
 found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyModule_NewObject: symbol not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyArg_UnpackTuple: symbol not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: _PyObject_GenericGetAttrWithDict: symbol 
not found
Error relocating cygrpc.cpython-39-arm-linux-gnueabihf.so: PyBytes_AsStringAndSize: symbol not found
...

I saw similar errors on the non-musl linux build w.r.t. the python relocation errors. Not sure if that is expected or is the issue. I'm not sure what are other best practice for checking out the shared objects.

lidizheng commented 2 years ago

Thanks for the update. I don't know off my head if CPython can be static linked. This (symbol-not-found error) should not happen if the CPython is installed. Because ARMv7 + musllinux is a niche combination, we might want to check if the given CPython on the device is built and installed for ARMv7 + musllinux.

allenporter commented 2 years ago

I've explicitly installed cython with apk add cython and it didn't seem to change the output from ld. I'm not actually sure which file it needs to link against, or how to see why the import is failing at runtime, etc.

To recap, here is the updated Dockerfile:

FROM arm32v7/alpine:3.14.2

RUN apk add --no-cache python3 py3-pip cython
RUN apk add libstdc++ libc6-compat libatomic musl-dev

WORKDIR /code
COPY example.py .

ENV WHEEL=grpcio-1.43.0.dev0-cp39-cp39-linux_armv7l.whl
COPY ${WHEEL} .

RUN pip3 install $WHEEL
RUN pip3 install google-cloud-pubsub

CMD python3 ./example.py

$ docker build -t local-wheel docker/local-wheel/
$ docker run local-wheel
Traceback (most recent call last):
  File "/code/./example.py", line 3, in <module>
    from google.cloud import pubsub_v1
  File "/usr/lib/python3.9/site-packages/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
    from google.cloud.pubsub_v1 import types
  File "/usr/lib/python3.9/site-packages/google/cloud/pubsub_v1/types.py", line 27, in <module>
    from google.api_core import gapic_v1
  File "/usr/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
    from google.api_core.gapic_v1 import config
  File "/usr/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/usr/lib/python3.9/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/usr/lib/python3.9/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: cannot import name 'cygrpc' from 'grpc._cython' (/usr/lib/python3.9/site-packages/grpc/_cython/__init__.py)

The above wheel was built with python3 tools/run_tests/task_runner.py -b python_linux_extra_armv7_cp39-cp39

Stepping back to the wheel generation process, my impression is that the existing arvm7 docker file (grpc_artifact_python_linux_armv7 install script)) does not succeed without modifications to get configure working:

PYTHON_HOST="armv7l-linux-musleabihf"
PYTHON_BUILD="x86"
printf "ac_cv_file__dev_ptmx=no\nac_cv_file__dev_ptc=no\n" > config.site
READELF=/usr/xcc/armv7-unknown-linux-gnueabi/bin/armv7-unknown-linux-gnueabi-readelf CONFIG_SITE="config.site" PYTHON_FOR_BUILD="${PYTHON_PREFIX}/bin/python-${PYTHON_VERSION}" ./configure --host="${PYTHON_HOST}" --build="${PYTHON_BUILD}" --disable-ipv6 
cp pyconfig.h "${PYTHON_PREFIX}"/include/python*

So perhaps two tracks:

Is there a known working way that grpc actually generates armv7 wheels at HEAD? (whether they work or not)
Now that we have a different error (before it was unable to find symbols at runtime) maybe I can look closer at the difference between the above alpine and arm32v7/python:3.8-buster w.r.t. ldd output and/or cython install.

allenporter commented 2 years ago

I was able to get an armv7 alpine wheel built for home assistant (codified here https://github.com/allenporter/wheels-grpc)

However, that wheel doesn't work on a plain Alpine install either and fails with the same error as the wheel built from grpc source. I think the hole assistant is building python from source and may be patching it in some way.
However my immediate user need is solved for the moment and we've been able to move to the latest grpc for the next release.

More investigation is needed here for a grpc built wheel that works on Alpine armv7

aptalca commented 2 years ago

@lidizheng

Just came across this issue with grpcio on armhf on a musl based distro (alpine).

The root issue seems to be that grpc is publishing a vanilla wheel (non-manylinux or musllinux) for armhf on pypi, which is only meant for glibc based systems, but because it is not container based and is vanilla, thus named grpcio-1.43.0-cp39-cp39-linux_armv7l.whl, pip cannot really tell whether it is compatible with the musl based system or not. It happily installs it, but the installed package does not work. Therefore anyone doing pip install grpcio on an alpine armhf system will have a broken package.

The x86_64 and aarch64 versions are fine, because they are actually published as manylinux on pypi, which is properly recognized by pip as not compatible with a musl based system and therefore pip correctly compiles from source.

I think the proper solution would be for grpc to not publish a vanilla wheel like that on pypi, and only rely on manylinux and musllinux (container based) wheels on linux (no idea how it is on other platforms like windows).

I'm not aware of any other project on pypi that publishes vanilla wheels.

Thanks

PS. If you build these wheels via pip wheel on armhf on ubuntu (glibc) and alpine (musl), both systems will produce wheels named grpcio-1.43.0-cp39-cp39-linux_armv7l.whl and pip cannot distinguish between the two, but they won't be compatible with each other. That's why it's best for official wheels on pypi to be container based as pip is aware of their compatibility and doesn't incorrectly use them on the wrong system.

We, at linuxserver.io, publish various vanilla wheels in our repos for reuse in our docker images, but in order to overcome the above mentioned issue, we keep glibc and musl based wheels in separate endpoints: https://wheel-index.linuxserver.io/ To complicate matters further, vanilla wheels also suffer from incompatibility due to system dependency package versions. For instance alpine 3.14 and 3.15 both have python 3.9 so pip thinks their vanilla wheels are compatible, but cffi for instance uses a shared library of libffi. If the cffi wheel is built on alpine 3.14, it is linked to libffi 7, but when it's used on alpine 3.15, it fails because 3.15 contains libffi 8. We can get away with maintaining separate repos for glibc vs musl, and different distro versions, because we can define different repos in our docker images, however for publishing on pypi, which is a single universal repo, the solution is to ship container based wheels with proper compatibility and with statically linked deps.

EDIT2: I should clarify that my above comments apply to cpython wheels. Pure python wheels (*-pyX-none-any.whl) are not glibc or musl dependent so they can be published as is on pypi with no issues.

gitpushdashf commented 2 years ago

Also running into this on Apple M1. x86_64 musllinux grpcio wheels were working great for me, but on M1 there's no wheel.

On Apple M1, the Debian-based ARM images can install grpcio from a wheel just fine. Alpine ARM would be nice, though!

vjp23 commented 1 year ago

Hi all, is there any movement on this issue? I've been bashing my head against the table trying to get GCP's Python client libraries working on an RPi 4 for months to no avail. :/

XuanWang-Amos commented 1 year ago

This is not a priority for us at the moment, but we'd be happy to assist anyone who wants to get involved.

grpc / grpc