Repairwheel breaks on musllinux (via cibuildwheel)

PiotrCzapla commented 1 day ago

Hi,

I've added the repairwheels step to setup.py and it is working fine. Unfrotunately your tool breaks when run via cibuildwheel on musllinux. I seems that you ship with audittool that only works for manylinux.

I've tested the audittool installed stand alone and it works as it ships with different set of platforms on musllinux, but it fails when run via repairwheel. Have a look,

/project # AUDITWHEEL_PLAT='musllinux_1_2_x86_64'  /opt/python/cp38-cp38/bin/python -m auditwheel repair /tmp/pywhispercpp-1.2.0-cp38-cp38-linux_x86_64.whl 
INFO:auditwheel.main_repair:Repairing pywhispercpp-1.2.0-cp38-cp38-linux_x86_64.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
INFO:auditwheel.wheeltools:New filename tags: musllinux_1_2_x86_64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp38-cp38-linux_x86_64
INFO:auditwheel.wheeltools:New WHEEL info tags: cp38-cp38-musllinux_1_2_x86_64
INFO:auditwheel.main_repair:
Fixed-up wheel written to /project/wheelhouse/pywhispercpp-1.2.0-cp38-cp38-musllinux_1_2_x86_64.whl

But it breaks when run via repairwheel:

/project # AUDITWHEEL_PLAT='musllinux_1_2_x86_64'  /opt/python/cp38-cp38/bin/python -m repairwheel /tmp/pywhispercpp-1.2.0-cp38-cp38-linux_x86_64.whl 
usage: __main__.py [-h] -o OUTPUT_DIR [-l LIB_DIR] [--no-sys-paths] [-V] wheel
__main__.py: error: the following arguments are required: -o/--output-dir
/project # AUDITWHEEL_PLAT='musllinux_1_2_x86_64'  /opt/python/cp38-cp38/bin/python -m repairwheel /tmp/pywhispercpp-1.2.0-cp38-cp38-linux_x86_64.whl  -o /tmp
Traceback (most recent call last):
  File "/opt/python/cp38-cp38/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/python/cp38-cp38/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/python/cp38-cp38/lib/python3.8/site-packages/repairwheel/__main__.py", line 4, in <module>
    main()
  File "/opt/python/cp38-cp38/lib/python3.8/site-packages/repairwheel/repair.py", line 106, in main
    fn(original_wheel, temp_wheel_dir, lib_path, use_sys_paths)
  File "/opt/python/cp38-cp38/lib/python3.8/site-packages/repairwheel/linux/repair.py", line 50, in repair
    main_repair.configure_parser(repair_sub_parsers)
  File "/opt/python/cp38-cp38/lib/python3.8/site-packages/repairwheel/_vendor/auditwheel/main_repair.py", line 43, in configure_parser
    p.add_argument(
  File "/opt/python/cp38-cp38/lib/python3.8/argparse.py", line 1368, in add_argument
    action = action_class(**kwargs)
  File "/opt/python/cp38-cp38/lib/python3.8/site-packages/repairwheel/_vendor/auditwheel/tools.py", line 122, in __init__
    raise argparse.ArgumentError(self, msg % args)
argparse.ArgumentError: argument --plat: invalid choice: 'musllinux_1_2_x86_64' from environment variable 'AUDITWHEEL_PLAT' (choose from 'linux_x86_64', 'manylinux_2_5_x86_64', 'manylinux_2_12_x86_64', 'manylinux_2_17_x86_64', 'manylinux_2_24_x86_64', 'manylinux_2_27_x86_64', 'manylinux_2_28_x86_64', 'manylinux_2_31_x86_64', 'manylinux_2_34_x86_64', 'manylinux_2_35_x86_64', 'manylinux1_x86_64', 'manylinux2010_x86_64', 'manylinux2014_x86_64')

to replicate this issue use docker 'musllinux_1_2_x86_64' docker run -it quay.io/pypa/musllinux_1_2_x86_64:2024.10.07-1 /bin/sh

Then pip install repair tool: /opt/python/cp38-cp38/bin/python -m pip install repairwheel

and try to run repairwheel with AUDITWHEEL_PLAT='musllinux_1_2_x86_64'

agriyakhetarpal commented 1 day ago

I remembered that there was some discussion about this over at https://github.com/ziglang/zig-pypi/issues/20#issuecomment-2088748050. Opening an issue here slipped out of my mind back then, so, thanks for this, @PiotrCzapla!

agriyakhetarpal commented 1 day ago

One (hacky) way to provide cross-platform support for this could be to ship[^1] the MUSL object files libc.musl-x86_64.so.1, libc.musl-aarch64.so.1, and those for other Linux architectures) within the repairwheel wheels as package data, and make them available when trying to repair a MUSL wheel.

This is on similar lines to what Zig does for its highly coveted cross-compilation support across platforms/architectures – as a white lie, it ships a copy of whatever libc is required to compile for a potential target: https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html

[^1]: I'm not sure where they can be sourced from, though, since I wasn't able to find pre-compiled distributions for them so far.

jvolkman / repairwheel

Repairwheel breaks on musllinux (via cibuildwheel) #43