search

python-greenlet / greenlet

Lightweight in-process concurrent programming
Other
1.63k stars 247 forks source link

Fix ABI tag in macOS wheels #372

Closed edgarrmondragon closed 10 months ago

edgarrmondragon commented 11 months ago

ABI tags look OK now:

Checking dist/greenlet-3.0.1.dev0-cp312-cp312-macosx_10_9_universal2.whl: PASSED

Closes #369

jamadden commented 11 months ago

Thanks for having a look. Unfortunately, the tags are very much not correct. You've re-introduced the exact problem the comment talked about solving: Most of the wheels lose their universal2 tag and become x86-only, even though that is not the case.

Archive:  greenlet-macOS-3.10.whl.zip
  inflating: greenlet-3.0.1.dev0-cp310-cp310-macosx_11_0_x86_64.whl
Archive:  greenlet-macOS-3.11.whl.zip
  inflating: greenlet-3.0.1.dev0-cp311-cp311-macosx_10_9_universal2.whl
Archive:  greenlet-macOS-3.12-dev.whl.zip
  inflating: greenlet-3.0.1.dev0-cp312-cp312-macosx_10_9_universal2.whl
Archive:  greenlet-macOS-3.7.whl.zip
  inflating: greenlet-3.0.1.dev0-cp37-cp37m-macosx_11_0_x86_64.whl
Archive:  greenlet-macOS-3.8.whl.zip
  inflating: greenlet-3.0.1.dev0-cp38-cp38-macosx_11_0_x86_64.whl
Archive:  greenlet-macOS-3.9.whl.zip
  inflating: greenlet-3.0.1.dev0-cp39-cp39-macosx_11_0_x86_64.whl
tasansal commented 11 months ago

@jamadden any idea why the current one can't be installed in Azure Pipelines with MacOS x86?

Poetry throws this; and it works on every other OS: Unable to find installation candidates for greenlet (3.0.0)

It does seem like there are two MacOS wheels one with cp311-cp311-universal2 and another cp311-universal2-universal2. Not sure if its related to this? image

jamadden commented 11 months ago

You'll need to look at pip's debug output, because it works fine for me on macOS. Maybe your pip is too old to understand universal2?

$ uname 
Darwin
$ arch -x86_64 /usr/local/bin/python3.11 -c 'import platform; print(platform.machine())'
x86_64
$ arch -x86_64 /usr/local/bin/python3.11 -m venv ./gtest
$ arch -x86_64 ./gtest/bin/python3.11 -c 'import platform; print(platform.machine())'
x86_64
$ arch -x86_64 ./gtest/bin/python3.11 -m pip install --no-cache-dir greenlet
Looking in indexes: https://pypi.org/simple, ...
Collecting greenlet
  Downloading greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl (262 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 262.9/262.9 kB 11.2 MB/s eta 0:00:00
Installing collected packages: greenlet
Successfully installed greenlet-3.0.0
$ arch -x86_64 ./gtest/bin/python3.11 -c 'import greenlet; print(greenlet.getcurrent())'
<greenlet.greenlet object at 0x10afc72a0 (otid=0x10afc71f0) current active started main>
edgarrmondragon commented 11 months ago

@jamadden Thanks for the feedback. So, I think the issue then is that the greenlet-3.0.0-cp3x-universal2-macosx_10_9_universal2.whl wheels are not installable and I noticed pip is instead installing the greenlet-3.0.0-cp3x-cp3x-macosx_10_9_universal2.whl wheels.

The problem then is that there's no 3.12 wheel for x86_64, but I don't know if that's expected:

$ http https://pypi.org/pypi/greenlet/json | jq '.urls[].filename' | grep universal
"greenlet-3.0.0-cp310-cp310-macosx_10_9_universal2.whl"
"greenlet-3.0.0-cp310-universal2-macosx_11_0_x86_64.whl"
"greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl"
"greenlet-3.0.0-cp311-universal2-macosx_10_9_universal2.whl"
"greenlet-3.0.0-cp312-universal2-macosx_10_9_universal2.whl"
"greenlet-3.0.0-cp37-universal2-macosx_11_0_x86_64.whl"
"greenlet-3.0.0-cp38-cp38-macosx_11_0_universal2.whl"
"greenlet-3.0.0-cp38-universal2-macosx_11_0_x86_64.whl"
"greenlet-3.0.0-cp39-cp39-macosx_10_9_universal2.whl"
"greenlet-3.0.0-cp39-universal2-macosx_11_0_x86_64.whl"

$ http https://pypi.org/pypi/greenlet/json | jq '.urls[].filename' | grep cp312
"greenlet-3.0.0-cp312-cp312-macosx_13_0_arm64.whl"
"greenlet-3.0.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"
"greenlet-3.0.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl"
"greenlet-3.0.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl"
"greenlet-3.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
"greenlet-3.0.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl"
"greenlet-3.0.0-cp312-cp312-musllinux_1_1_aarch64.whl"
"greenlet-3.0.0-cp312-cp312-musllinux_1_1_x86_64.whl"
"greenlet-3.0.0-cp312-cp312-win_amd64.whl"
"greenlet-3.0.0-cp312-universal2-macosx_10_9_universal2.whl"

This also affects the wheel for 3.7 but probably fewer people will notice since it's EOL anyway.

Details # 3.11 ### `x86_64` ```console $ uname Darwin $ arch -x86_64 /usr/local/bin/python3.11 -c 'import platform; print(platform.machine())' x86_64 $ arch -x86_64 /usr/local/bin/python3.11 -m venv ./gtest $ arch -x86_64 ./gtest/bin/python3.11 -c 'import platform; print(platform.machine())' x86_64 $ arch -x86_64 ./gtest/bin/python3.11 -m pip install --upgrade pip Requirement already satisfied: pip in ./gtest/lib/python3.11/site-packages (23.2.1) $ arch -x86_64 ./gtest/bin/python3.11 -m pip install --no-cache-dir greenlet Collecting greenlet Obtaining dependency information for greenlet from https://files.pythonhosted.org/packages/a0/fb/3eeb54137cc4d01248babb62fd12c7a5faba36b13692ed622ea43fd4d648/greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl.metadata Downloading greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl.metadata (3.8 kB) Downloading greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl (262 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 262.9/262.9 kB 1.5 MB/s eta 0:00:00 Installing collected packages: greenlet Successfully installed greenlet-3.0.0 ``` ### `arm64` ```console $ uname Darwin $ /usr/local/bin/python3.11 -c 'import platform; print(platform.machine())' arm64 $ /usr/local/bin/python3.11 -m venv ./gtest $ ./gtest/bin/python3.11 -c 'import platform; print(platform.machine())' arm64 $ ./gtest/bin/python3.11 -m pip install --upgrade pip Requirement already satisfied: pip in ./gtest/lib/python3.11/site-packages (23.2.1) $ ./gtest/bin/python3.11 -m pip install --no-cache-dir greenlet Collecting greenlet Obtaining dependency information for greenlet from https://files.pythonhosted.org/packages/a0/fb/3eeb54137cc4d01248babb62fd12c7a5faba36b13692ed622ea43fd4d648/greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl.metadata Downloading greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl.metadata (3.8 kB) Downloading greenlet-3.0.0-cp311-cp311-macosx_10_9_universal2.whl (262 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 262.9/262.9 kB 1.3 MB/s eta 0:00:00 Installing collected packages: greenlet Successfully installed greenlet-3.0.0 ``` --- ## 3.12 ### `x86_64` ```console $ uname Darwin $ arch -x86_64 /usr/local/bin/python3.12 -c 'import platform; print(platform.machine())' x86_64 $ arch -x86_64 /usr/local/bin/python3.12 -m venv ./gtest $ arch -x86_64 ./gtest/bin/python3.12 -c 'import platform; print(platform.machine())' x86_64 $ arch -x86_64 ./gtest/bin/python3.12 -m pip install --upgrade pip Requirement already satisfied: pip in ./gtest/lib/python3.12/site-packages (23.1.2) Collecting pip Using cached pip-23.2.1-py3-none-any.whl (2.1 MB) Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 23.1.2 Uninstalling pip-23.1.2: Successfully uninstalled pip-23.1.2 Successfully installed pip-23.2.1 $ arch -x86_64 ./gtest/bin/python3.12 -m pip install --no-cache-dir greenlet Collecting greenlet Downloading greenlet-3.0.0.tar.gz (174 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 174.7/174.7 kB 1.3 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: greenlet Building wheel for greenlet (pyproject.toml) ... done Created wheel for greenlet: filename=greenlet-3.0.0-cp312-cp312-macosx_10_9_universal2.whl size=264198 sha256=25ff12c4fd7ddc54b81153b23ac02e2afed3a673623a3f00e0de3671a30ff3eb Stored in directory: /private/var/folders/k7/l2cj2kv14x9dg7ljk_p20y7w0000gn/T/pip-ephem-wheel-cache-6prw073j/wheels/dc/eb/db/f1e5d3f955b72b286f756c7af246929307f37b2e38defd8fcb Successfully built greenlet Installing collected packages: greenlet Successfully installed greenlet-3.0.0 ``` ### `arm64` ```console $ uname Darwin $ /usr/local/bin/python3.12 -c 'import platform; print(platform.machine())' arm64 $ /usr/local/bin/python3.12 -m venv ./gtest $ ./gtest/bin/python3.12 -c 'import platform; print(platform.machine())' arm64 $ ./gtest/bin/python3.12 -m pip install --upgrade pip Requirement already satisfied: pip in ./gtest/lib/python3.12/site-packages (23.1.2) Collecting pip Using cached pip-23.2.1-py3-none-any.whl (2.1 MB) Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 23.1.2 Uninstalling pip-23.1.2: Successfully uninstalled pip-23.1.2 Successfully installed pip-23.2.1 $ ./gtest/bin/python3.12 -m pip install --no-cache-dir greenlet Collecting greenlet Obtaining dependency information for greenlet from https://files.pythonhosted.org/packages/01/df/8ce8c5a549fcd4b61d9297dcbdb541745ca7a0b2b9b62eec48b8a3db1dc5/greenlet-3.0.0-cp312-cp312-macosx_13_0_arm64.whl.metadata Downloading greenlet-3.0.0-cp312-cp312-macosx_13_0_arm64.whl.metadata (3.8 kB) Downloading greenlet-3.0.0-cp312-cp312-macosx_13_0_arm64.whl (216 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 216.9/216.9 kB 1.5 MB/s eta 0:00:00 Installing collected packages: greenlet Successfully installed greenlet-3.0.0 ```
layday commented 11 months ago

In CI, you have/had:

wheel tags --remove --abi-tag universal2 dist/*whl

I don't see what this accomplishes. Candidates for the targeted architecture are chosen based on the platform tag, not the ABI tag.