Request: Prebuilt musllinux arm artifacts

[MattDynamite]

What version of gRPC and what language are you using?

I am trying to install grpcio==1.59.2 (But it happened with any other version I used) and python 3.8.

What operating system (Linux, Windows,...) and version?

I'm using Linux, to be more specific a docker image of python 3.8 alpine linux based.

What runtime / compiler are you using (e.g. python version or version of gcc)

python 3.8, I install gcc using apk add at the beginning of my Dockerfile.

What did you do?

Please provide either 1) A unit test for reproducing the bug or 2) Specific steps for us to follow to reproduce the bug. If there’s not enough information to debug the problem, gRPC team may close the issue at their discretion. You’re welcome to re-open the issue once you have a reproduction.

Here's a minimal Dockerfile to build in order to reproduce the issue (Running on MAC M1 arm64 originally):

FROM python:3.8-alpine

RUN apk --update --no-cache add \
    zlib-dev \
    musl-dev \
    libc-dev \
    libffi-dev \
    openssh-client \
    jq \
    gcc \
    g++ \
    make \
    git \
    pwgen \
    linux-headers \
    && pip install --upgrade pip

RUN python3 -m pip install --upgrade setuptools wheel

RUN pip install --no-cache-dir grpcio==1.59.2

Then run: docker build -t my-docker-minmal-alpine:latest --platform=linux/arm64 -f docker/Dockerfile .

After the first steps the collection of grpcio will start:

 RUN pip install --no-cache-dir grpcio==1.59.2                                                                                                                                                                                                                                                       5.9s
 => => # Collecting grpcio==1.59.2                                                                                                                                                                                                                                                                                
 => => #   Downloading grpcio-1.59.2.tar.gz (24.8 MB)                                                                                                                                                                                                                                                             
 => => #      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.8/24.8 MB 450.9 kB/s eta 0:00:00                                                                                                                                                                                                                         
 => => #   Preparing metadata (setup.py): started                                                                                                                                                                                                                                                                 
 => => #   Preparing metadata (setup.py): finished with status 'done'                                                                                                                                                                                                                                             
 => => # Building wheels for collected packages: grpcio                                                                                                                                                                                                                                                           
 => => #   Building wheel for grpcio (setup.py): started

And will be stuck for a long while. The reason for that is that there is no matching wheel for this architecture and base image:

#8 1.236 Collecting grpcio==1.59.2
#8 1.435   Downloading grpcio-1.59.2.tar.gz (24.8 MB)
#8 4.310      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.8/24.8 MB 9.8 MB/s eta 0:00:00
#8 5.644   Preparing metadata (setup.py): started
#8 5.944   Preparing metadata (setup.py): finished with status 'done'
#8 5.947 Building wheels for collected packages: grpcio
#8 5.947   Building wheel for grpcio (setup.py): started
#8 66.15   Building wheel for grpcio (setup.py): still running...
#8 126.1   Building wheel for grpcio (setup.py): still running...
#8 186.2   Building wheel for grpcio (setup.py): still running...
#8 249.2   Building wheel for grpcio (setup.py): still running...
#8 249.3   Building wheel for grpcio (setup.py): finished with status 'done'
#8 249.4   Created wheel for grpcio: filename=grpcio-1.59.2-cp38-cp38-linux_aarch64.whl

What did you expect to see?

I expected the docker build command to work successfully and install the python library.

What did you see instead?

The docker build command took a very long time before reaching timeout in external runner (It succeeded on my local machine but it's important to note that this machine is a very strong one).

Anything else we should know about your project / environment?

It seems like a compatiblity issue with musl and linux/arm64. I took a look at the release notes of grpcio==1.59.2 and found a whl that should be compatible with the architecture and tried to install it directly instead but to no-avail. Guess you just need to also create a wheel for this specific arch: grpcio-1.59.2-cp38-cp38-linux_aarch64.whl

[XuanWang-Amos]

Hi, we currently don't have wheels for musllinux aarch64, this will require us some time work on it.

It's not a priority for us know, but for anyone who's checking on this issue, please leave a comment so we can re-evaluate the priority.

[omenking]

While attempting to install pulumni_aws I ran into this same issue with grpcio hanging forever.

[HassamK97]

I am facing the same issue, installing certain libraries has dependency of grpcio like I am only trying to install firebase and it requires grpcio. I have python 3.8.10 running on Vim3 Ubuntu 20.04, it's an aarch64 based linux kernal.

[iskhakov]

We run into this issue too, please add musllinux aarch64 wheels

[bconway]

As a workaround, you could use the packages built/included with Alpine. I've had luck with protobuf-dev grpc-plugins on Alpine arm64.

[SchrodingersGat]

@XuanWang-Amos also running into this issue - https://github.com/inventree/InvenTree/issues/6618

@bconway can you provide some more context for your suggestion? How do you make use of protobuf-dev / grpc-plugins?

[bconway]

My builds are spread between Dockerfiles and Makefiles, but the relevant pieces are along the lines of:

apk add --no-cache git make protobuf-dev grpc-plugins
protoc --plugin=protoc-gen-grpc=/usr/bin/grpc_python_plugin -I=../protobuf \
--python_out=. --pyi_out=. --grpc_out=. api/alert.proto

Works well on Alpine arm64. The same code is generated, because (I believe) grpcio is wrapping the same library or binary.

[khodedawsh]

Same problem here. Tried waiting for it to build but looks like it takes forever.

[graphenn]

We run into this issue too, build grpcio and grpcio-tools takes too long time, please add musllinux aarch64 wheels

[andrewrmiller]

We ran into this issue too.

[Olegt0rr]

Same problem

macOS (M1 Pro) Docker python:3.11-alpine

 => => #   Building wheel for grpcio (pyproject.toml): started                                                                                                                                             
 => => #   Building wheel for grpcio (pyproject.toml): still running...  

[cyphx]

Same problem, has anyone tried building a wheel for musllinux for arm64 architecture. Trying to run a project locally on mac m1 which requires grpcio as one of its dependencies. Build keeps on running until build timeout

[tomasz-utracki-janeta]

Same issue here, attempt to build on Mac M1 fails with timeout.

[sourabhsinghs]

Anyone who's checking on this issue, please leave an upvote or comment, so we can re-evaluate the priority for this.

[dienhartd]

Similarly, on M1 Max Macbook Pro ran a poetry install with grpcio as a dependency for a Docker python:3.11-alpine image build. Watching with docker build --progress=plain showed that any version fails to build wheel after a long series of g++ build commands.