search

pypa / auditwheel

Auditing and relabeling cross-distribution Linux wheels.
Other
431 stars 142 forks source link

RISC-V support #408

Open jpruciak opened 1 year ago

jpruciak commented 1 year ago

When running auditwheel on linux_riscv64 wheel it crashes.

[189:24] Fixing wheelhouse/cryptography-38.0.4-cp311-cp311-linux_riscv64.whl
[189:24] AssertionError: [{'name': 'linux_riscv64', 'aliases': [], 'priority': 0, 'symbol_versions': {}, 'lib_whitelist': [], 'blacklist': {}}]
[189:26] Traceback (most recent call last):
[189:26] File "/builds/funkwhale/funkwhale/tools/bin/auditwheel", line 5, in <module>
[189:26] from auditwheel.main import main
[189:26] File "/builds/funkwhale/funkwhale/tools/lib/python3.11/site-packages/auditwheel/main.py", line 16, in <module>
[189:26] from . import main_addtag, main_lddtree, main_repair, main_show
[189:26] File "/builds/funkwhale/funkwhale/tools/lib/python3.11/site-packages/auditwheel/main_repair.py", line 7, in <module>
[189:26] from .policy import (
[189:26] File "/builds/funkwhale/funkwhale/tools/lib/python3.11/site-packages/auditwheel/policy/__init__.py", line 118, in <module>

It looks like policies for musllinux and manylinux are missing.

jpruciak commented 1 year ago

This diff fixes it for musllinux, I've not worked on getting manylinux working.

diff --git a/src/auditwheel/policy/__init__.py b/src/auditwheel/policy/__init__.py
index a1a30ca..274cbcd 100644
--- a/src/auditwheel/policy/__init__.py
+++ b/src/auditwheel/policy/__init__.py
@@ -88,6 +88,7 @@ def _fixup_musl_libc_soname(whitelist):
             "s390x": "libc.musl-s390x.so.1",
             "ppc64le": "libc.musl-ppc64le.so.1",
             "armv7l": "libc.musl-armv7.so.1",
+            "riscv64": "libc.musl-riscv64.so.1",
         }
     }
     new_whitelist = []
diff --git a/src/auditwheel/policy/musllinux-policy.json b/src/auditwheel/policy/musllinux-policy.json
index b103099..f2f6a06 100644
--- a/src/auditwheel/policy/musllinux-policy.json
+++ b/src/auditwheel/policy/musllinux-policy.json
@@ -21,6 +21,8 @@
       "s390x": {
       },
       "armv7l": {
+      },
+      "riscv64": {
       }
     },
     "lib_whitelist": ["libc.so", "libz.so.1"],
@@ -42,6 +44,8 @@
       "s390x": {
       },
       "armv7l": {
+      },
+      "riscv64": {
       }
     },
     "lib_whitelist": ["libc.so", "libz.so.1"],
lkollar commented 1 year ago

This would first need a manylinux image, so I suggest opening an issue there. That will need some sort of public CI system which has RISC-V runners. I don't think Travis or GH Actions supports RISC-V at the moment, but I could be wrong. These would be the minimum requirements to add support for the platform.

JuniorJPDJ commented 1 year ago

RISC-V support can be enabled using qemu-user-static based actions for runners on github actions eg. using this: https://github.com/docker/setup-qemu-action

AFAIK this is the same way as running armv7 on actions.

JuniorJPDJ commented 1 year ago

Also I think that manylinux image is totally another problem, auditwheel can support repairing wheels but without official support from manylinux images side. When they will decide to include RISC-V compatible image for manylinux (Alpine stable still doesn't support riscv, so musllinux is no way atm anyway) - the path will be clear as auditwheel would support it already.

JuniorJPDJ commented 1 year ago

Anyway I created the issue: https://github.com/pypa/manylinux/issues/1425

lkollar commented 1 year ago

Also I think that manylinux image is totally another problem, auditwheel can support repairing wheels but without official support from manylinux images side.

We use the manylinux images in the test suite, and without a riscv64 image we can't test if auditwheel is working correctly on this platform. Thanks for filing the issue in manylinux.

mayeut commented 1 year ago

We use the manylinux images in the test suite, and without a riscv64 image we can't test if auditwheel is working correctly on this platform.

I think this could be added as an untested ALPHA feature. Anyway, we'll always be in a chicken & egg situation (manylinux requires auditwheel, we can't have an image without support in auditwheel).

As far as I could see:

I updated https://github.com/mayeut/pep600_compliance/commit/4b6a18a885d76db5e3ec390efeb834774dccc591 and added some manylinux policies in there.

I'm open to a PR adding those policies & the bits missing in auditwheel as long as support is marked experimental if @lkollar agrees.

@JuniorJPDJ, you'll want to make sure pip gets proper support for this as well (through packaging). As of now, this is not the case. PyPI will also refuse riscv64 uploads as of now.

As it involves multiple python packaging projects, I advise you to create a discussion at https://discuss.python.org/c/packaging/14

JuniorJPDJ commented 1 year ago

I was talking with alpine maintainers about riscv support and they said as long as they do not have server-grade riscv boards for builders it's no go for stable. Edge/rolling supports RISC-V for a while already. Seeing this supported as experimental would be good enough for me, as I just want to build packages for private repositories in Funkwhale project.

eshattow commented 1 month ago

https://wiki.alpinelinux.org/wiki/Riscv64

RISC V's first release into Alpine Linux was in v3.20.