kevinheavey / solders

A high-performance Python toolkit for Solana, written in Rust
https://kevinheavey.github.io/solders/
Apache License 2.0
232 stars 26 forks source link

0.20.0 aarch64 wheels on PyPi seem to be bad #86

Closed Polsaker closed 8 months ago

Polsaker commented 8 months ago

Hello, testing this lib on an ARM machine I'm getting the following error (this is not happening on an x86 container):

  File "/usr/local/lib/python3.12/site-packages/solana/rpc/api.py", line 11, in <module>
    from solders.rpc.responses import (
ModuleNotFoundError: No module named 'solders.rpc.responses'

Checking the wheel files on PyPI you can notice that the aarch64 wheels are suspiciously smaller than the wheels for other archs:

image

kevinheavey commented 8 months ago

Unfortunately this is deliberate because the required dependencies (specifically ring) don't compile on linux aarch64, so we leave them out:

https://github.com/kevinheavey/solders/blob/b14a24010b1dd2e99fb5011e4e4503d3789a1e9c/.github/workflows/build.yml#L248C16-L248C59

I just triggered a CI run that tries to include the ring dependency so you can see https://github.com/kevinheavey/solders/actions/runs/8264519529/job/22608292459?pr=87

Even more annoying is that we don't even need that code, but a bunch of Solana crates depend on reqwest when they really shouldn't. For example solana-account-decoder depends on solana-config-program which indirectly depends on reqwest :(

Polsaker commented 8 months ago

Maybe it would be better not to offer aarch64 wheels then? So pip install can at least attempt to build locally by default.

For us forcing pip to build locally worked just fine (it just added a few minutes to all our pipelines 🙃)

kevinheavey commented 8 months ago

Oh that's weird that it compiles for you on Linux aarch64 right? Since it doesn't compile in CI

Polsaker commented 8 months ago

If it helps at all our CI runs on aws graviton boxes and for this particular case in a container with the python:3.12 image where we installed stable-aarch64-unknown-linux-gnu rust 1.76.0. Then we just did a poetry install and it worked

kevinheavey commented 8 months ago

Thanks for the info, that led me to discover there actually was a way to get ring to compile in CI. Fixed by #87