search

openvinotoolkit / openvino_tokenizers

OpenVINO Tokenizers extension
Apache License 2.0
18 stars 16 forks source link

[WHEEL] Freeze scikit-build-core version #123

Closed mryzhov closed 4 months ago

mryzhov commented 4 months ago

The new scikit-build-core version skips manylinux tag as one of the valid wheels tags:

https://github.com/scikit-build/scikit-build-core/pull/698

henryiii commented 4 months ago

This was being incorrectly selected (due to what I believe is a bug in packaging - It's https://github.com/pypa/packaging/issues/160, sadly closed in favor of making a new tag that never got made, but producing many/musl linux wheels without auditing them is clearly incorrect, and the issue specifically mentions that these must be filtered by packaging tools). The wheels are not guaranteed to be manylinux wheels until you run them through auditwheel, which verifies they don't use anything that is not part of the manylinux spec, and bundles any dependencies not available in manylinux into the wheel with name mangling. Until that is run, the wheel is linux, which is not redistributable. manylinux ensures the user that this process has been run.

cibuildwheel does all this for you, or you can do it manually. If you really want to fake it, you can use pipx run wheel tags --platform-tag manylinux_2_17_x86_64.manylinux2014_x86_64 *.whl (or whatever manylinux tag you want) to change the tag manually. You can pass lots of different flags to control auditwheel repair, though, so that is highly recommended instead of just retagging.

I've never used it, but there's also https://github.com/jvolkman/repairwheel, which bundles auditwheel.

henryiii commented 4 months ago

The correct fix is to run auditwheel to produce the manylinux wheels, like this:

$ auditwheel repair openvino_tokenizers-2024.1.0.0-81-py3-none-linux_x86_64.whl --exclude libopenvino_tensorflow_frontend.so.2410 --exclude libopenvino.so.2410
INFO:auditwheel.main_repair:Repairing openvino_tokenizers-2024.1.0.0-81-py3-none-linux_x86_64.whl
INFO:auditwheel.lddtree:Excluding libopenvino_tensorflow_frontend.so.2410
INFO:auditwheel.lddtree:Excluding libopenvino.so.2410
INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_x86_64, manylinux2014_x86_64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: py3-none-linux_x86_64
INFO:auditwheel.wheeltools:New WHEEL info tags: py3-none-manylinux_2_17_x86_64, py3-none-manylinux2014_x86_64
INFO:auditwheel.main_repair:
Fixed-up wheel written to /wheelhouse/openvino_tokenizers-2024.1.0.0-81-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

See https://github.com/scikit-build/scikit-build-core/pull/698#issuecomment-2085759622 for more info. This is more secure, as it protects you if any other things that are non-compliant sneak in over time. (You can also use cibuildwheel; with cibuildhweel, you can override the repair command to add the excludes above).