search

rasterio / rasterio-wheels

MIT License
11 stars 16 forks source link

GitHub Actions for some wheel builds #73

Closed sgillies closed 2 years ago

sgillies commented 2 years ago

I'm going to try following the example of python-pillow/pillow-wheels, which runs multibuild on GHA https://github.com/python-pillow/pillow-wheels/actions/runs/1655656052. @snowman2 I think I recall you suggesting this?

Pillow wheels for aarch64 only are built on Travis: https://app.travis-ci.com/github/python-pillow/pillow-wheels.

snowman2 commented 2 years ago

I don't remember suggesting it, but it sounds like a great idea. Looks like they have a nice setup: https://github.com/python-pillow/pillow-wheels/blob/main/.github/workflows/wheels.yml

vincentsarago commented 2 years ago

@sgillies I know we might not want to change everything but https://github.com/pypa/cibuildwheel seems a nice way to build wheels too

example: https://github.com/kylebarron/pymartini/blob/master/.github/workflows/build-wheels.yml

kylebarron commented 2 years ago

Vincent mentioned this issue to me. I've been happy with cibuildwheel's support for lots of different targets, including Apple Silicon and even musllinux for Alpine Linux. I understand that rasterio's build process is very complex, but evaluating cibuildwheel might make sense. 🤷‍♂️

Edit: Looks like lots of projects use it, so it could be worth looking at a project with similar complexity as rasterio for inspiration https://cibuildwheel.readthedocs.io/en/stable/working-examples/

sgillies commented 2 years ago

I've never seen a project with complexity similar to rasterio :rofl: We include openssl, libcurl, and nghttp2 because GDAL is almost an entire operating system.

sgillies commented 2 years ago

It's sorta working (linux only): https://github.com/rasterio/rasterio-wheels/runs/4721671335! I expect to hit a time limit, though. Even with make -j4 it takes nearly an hour to compile everything.

sgillies commented 2 years ago

No timeouts! And 2 jobs succeeded in making tested wheels. That's an encouraging start :sweat_smile:

sgillies commented 2 years ago

I'm stuck on the macos arm64 jobs, but we've got wheels for manylinux 2014 and macos x86_64, Pythons 3-7-3.10: https://github.com/rasterio/rasterio-wheels/runs/4734505845.

sgillies commented 2 years ago

macosx arm64 are blocked and require some serious work to get around issues in hdf5 (see #75). The other 8 jobs are fine.

vincentsarago commented 2 years ago

Note: if we ever want to move to cibuildwheel, I found a nice feature used in asyncpg and many other project: https://github.com/MagicStack/asyncpg/blob/220d1d508a90a7db2a8172348561406a9316c3ca/pyproject.toml#L8-L24

It seems you can easily run scripts for each env