Closed snowman2 closed 1 year ago
I built wheel for pyproj 3.3.1 for arm64 python3.9 and python3.10. If you teel me how put on pypip I Will do with pleasure. I test these wheel with pytest and the tests are all passed
@vot4anto, there are a couple of ways to do it:
Unfortunately our GitHub Action ARM64 runners are not always online. Since now i have to poweron It to use, I still work how use a scripts with MacVm to create It on demand. Follow link to the wheels and you can download .
https://wheelhouse.openquake.org/macos/arm64/py39/pyproj-3.3.1-cp39-cp39-macosx_11_0_arm64.whl
And
https://wheelhouse.openquake.org/macos/arm64/py310/pyproj-3.3.1-cp310-cp310-macosx_11_0_arm64.whl
Hope these wheels works fine, with pur OpenQuake engine works as expected
First of all, thank you for sharing links to the wheels you uploaded :+1:
I looked at the wheels recently uploaded to pypi for x86_64
and it has these libraries included:
This is the script we use to prepare the wheels for MacOS: https://github.com/pyproj4/pyproj/blob/main/ci/proj-compile-wheels.s
These are the libraries included in the wheels you provided:
You mentioned that everything worked for you as expected. I wanted to clarify and ask if you tested the PROJ Network capabilities with the wheels you generated?
Hi, I use this script to build the wheel: https://github.com/gem/oq-builders/blob/master/.github/workflows/pyproj_cibuild_m1.yml
And essentially I copy your script and use the source of 3.3.1 to build the wheel. Unfortunately there was a lot of failure due the configuration of the runner, for example the configuration of sudo for user runner and other things like that and always I need to recreate from scratch the VM of the runner
I run again the actions and I have on the wheel the same contents, I also see the follows in the test section of cibuildwheel:
test/test_transformer.py::test_network__disable[transformer0] PASSED
test/test_transformer.py::test_network__disable[transformer1] PASSED
test/test_transformer.py::test_network__disable[transformer2] PASSED
test/test_transformer.py::test_network__enable[transformer0] PASSED
test/test_transformer.py::test_network__enable[transformer1] PASSED
test/test_transformer.py::test_network__enable[transformer2] PASSED
test/test_transformer.py::test_network__default[transformer0] PASSED
test/test_transformer.py::test_network__default[transformer1] PASSED
test/test_transformer.py::test_network__default[transformer2] PASSED
@snowman2 Are those the test that you mention?
Looks good. It looks like you are testing with PROJ_NETWORK=ON
, so I think it should be good.
Would you mind adding PROJ 8.2
to the wheels? It is what the other wheels were build with for the 3.3.1 release (https://github.com/pyproj4/pyproj/discussions/1063).
I compile the wheel with the env variabile PROJ=8.2.1
in first versione of wheel and after with PROJ=9.0.0
It Is enough? Or I need to
change something in the scripts to adding PROJ 8.2
?
PROJ_VERSION was 8.2.0 for the 3.3.1 wheels. It would be nice to keep it consistent.
When the 3.4.0 pyproj release happens, the wheels will contain the latest version of PROJ at the time of the release.
I set PROJ_VERSION to 8.2.0 and recreate the wheels attached
Thanks @vot4anto 👍. They are on pypi now.
Does that mean that we should be able to install pyproj by pip downloading pre-built wheels? I have a M1 Macbook, and can not get it to install without further requirements:
$ python --version
Python 3.10.6
$ python -m pip install pyproj
Collecting pyproj
Using cached pyproj-3.4.0.tar.gz (217 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
proj executable not found. Please set the PROJ_DIR variable. For more information see: https://pyproj4.github.io/pyproj/stable/installation.html
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
After installing proj with brew install proj
, it did work:
$ python -m pip install pyproj
Collecting pyproj
Using cached pyproj-3.4.0.tar.gz (217 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting certifi
Downloading certifi-2022.9.14-py3-none-any.whl (162 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 3.3 MB/s eta 0:00:00
Building wheels for collected packages: pyproj
Building wheel for pyproj (pyproject.toml) ... done
Created wheel for pyproj: filename=pyproj-3.4.0-cp310-cp310-macosx_12_0_arm64.whl size=413134 sha256=dabb12412967ba21fc4d88731a7ff189337034cb9240470842ef322c85435572
Stored in directory: /Users/user/Library/Caches/pip/wheels/88/9b/b4/515b18b3dcb8e3af3b25ed03a391e4ba0f0246fa7d465dc9e5
Successfully built pyproj
Installing collected packages: certifi, pyproj
Successfully installed certifi-2022.9.14 pyproj-3.4.0
I'm sorry but I have some trouble and can't have time to build whl for 3.4.0. I will do asap
cibuildwheel project now add M1 support with Cirrus CI https://github.com/pypa/cibuildwheel Try to use it with fork of proj 3.4.0 since I have test failed with old method:
test/test_sync.py::test_get_transform_grid_list__contains FAILED
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test/test_sync.py::test_get_transform_grid_list__contains - AssertionE...
============ 1 failed, 838 passed, 1 skipped, 47 warnings in 18.51s ============
If I compile the wheel from main I don't have any errors, instead the build faild on tag and new branch https://github.com/vot4anto/pyproj/tree/3.4.0-arm64 that I have created to built the wheel. As you can see from the follow: Failed: https://github.com/vot4anto/pyproj/actions/runs/3077093832 Success: https://github.com/vot4anto/pyproj/actions/runs/3076994603
In the successfull build there are the wheel for python3.9, 3.10:
Length Date Time Name
--------- ---------- ----- ----
3792707 2022-09-18 11:24 pyproj-3.4.1.dev0-cp310-cp310-macosx_11_0_arm64.whl
3798222 2022-09-18 11:24 pyproj-3.4.1.dev0-cp39-cp39-macosx_11_0_arm64.whl
Unfortunately with cibuildwheel@v2.8.0 don't compile python 3.11 and with higher version I have this error:
Version 3.7 - 3.10 was not found in the local cache
Error: Version 3.7 - 3.10 with arch arm64 not found
The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
The test failure is okay - see #1145. You can safely skip it.
If you update to cibuildwheel 2.9, you should be able to build Python 3.11 wheels.
I can build the wheel for python 3.9 and 3.10 after you suggestion and attach to this comment. Unfortunately if I use cibuildwheel 2.9 or 2.10 the all process fail with the follow error:
Run actions/setup-python@v4
Version 3.7 - 3.10 was not found in the local cache
Error: Version 3.7 - 3.10 with arch arm64 not found
As you can see: https://github.com/vot4anto/pyproj/actions/runs/3080187090/jobs/4977249167
I think something is changed from those release of cibuildwheel
I change the workflow and now I can use also python 3.11 But the test of the wheel fails for python 3.11
==================================== ERRORS ====================================
______________________ ERROR collecting test/test_geod.py ______________________
test/test_geod.py:18: in <module>
from shapely.geometry import (
/private/var/folders/jd/2kp_brz51hl61lhbbsxp9dy80000gn/T/cibw-run-gefupi54/cp311-macosx_arm64/venv-test/lib/python3.11/site-packages/shapely/geometry/__init__.py:4: in <module>
from .base import CAP_STYLE, JOIN_STYLE
/private/var/folders/jd/2kp_brz51hl61lhbbsxp9dy80000gn/T/cibw-run-gefupi54/cp311-macosx_arm64/venv-test/lib/python3.11/site-packages/shapely/geometry/base.py:20: in <module>
from shapely.coords import CoordinateSequence
/private/var/folders/jd/2kp_brz51hl61lhbbsxp9dy80000gn/T/cibw-run-gefupi54/cp311-macosx_arm64/venv-test/lib/python3.11/site-packages/shapely/coords.py:10: in <module>
from shapely.geos import lgeos
/private/var/folders/jd/2kp_brz51hl61lhbbsxp9dy80000gn/T/cibw-run-gefupi54/cp311-macosx_arm64/venv-test/lib/python3.11/site-packages/shapely/geos.py:164: in <module>
_lgeos = load_dll('geos_c', fallbacks=alt_paths)
/private/var/folders/jd/2kp_brz51hl61lhbbsxp9dy80000gn/T/cibw-run-gefupi54/cp311-macosx_arm64/venv-test/lib/python3.11/site-packages/shapely/geos.py:54: in load_dll
raise OSError(
E OSError: Could not find lib geos_c or load any of its variants ['/Library/Frameworks/GEOS.framework/Versions/Current/GEOS', '/opt/local/lib/libgeos_c.dylib', '/usr/local/lib/libgeos_c.dylib', '/opt/homebrew/lib/libgeos_c.dylib'].
=============================== warnings summary ===============================
https://github.com/vot4anto/pyproj/actions/runs/3080277927/jobs/4977431916
Thanks @vot4anto :+1:
For the 3.11 wheels, you should be able to skip installing shapely and the tests should skip the tests requiring shapely.
Ok, I will update the workflow to skip installation only on python 3.11 and I will upload the wheel asap
cibuildwheel project now add M1 support with Cirrus CI https://github.com/pypa/cibuildwheel
That is good to know, thanks :+1:. Hopefully we can get that setup so we can automate this.
As suggested I skip installation of shapely and the wheel are created also for 3.11
Thanks @vot4anto :+1:
@snowman2 FYI I just added MacOS arm64 wheels to pyogrio's cibuildwheel setup (using github actions and cross-compiling, so without being able to run tests (but that's also what eg conda-forge does)): https://github.com/geopandas/pyogrio/pull/64 (if this works for GDAL, it should also work for PROJ ;))
Thanks for the update @jorisvandenbossche :+1:
Currently don't have CI runners to build/test them. There are options to potentially cross compile, but the wheels wouldn't be able to be tested. Not sure what the best way to handle this is.
Related: