rasterio / rasterio-wheels

MIT License
11 stars 16 forks source link

First try at Cirrus config #99

Closed sgillies closed 1 year ago

sgillies commented 1 year ago

Getting builds going on a new platform is always an epic quest. A bunch of GDAL's dependencies like openssl and curl built, and then I get to libpng and install: /Users/admin/local/share/man/man3/libpng.3: Permission denied. https://cirrus-ci.com/task/5011115680202752?logs=build_wheel#L4811. WTF?

sgillies commented 1 year ago

I made some progress today! This project's config.sh script had some missing PREFX in builds, and /usr/local is not as free to use as it used to be on the latest Macs. I've progressed to the point of building GDAL itself, which is failing because GDAL 3.5.3's configure script emits a broken -arch arm64-arch arm64 configuration on arm-apple-darwin. To go forward, it looks like we have to switch to using cmake to build GDAL.

@rouault can I bug you for advice? Does GDAL 3.5.3 build cleanly on Apple M1 using Cmake?

rouault commented 1 year ago

Does GDAL 3.5.3 build cleanly on Apple M1 using Cmake?

Seeing https://github.com/OSGeo/gdal/issues/5994 I assume so

rouault commented 1 year ago

The build problem of the OGCAPI driver is related to the lack of the GML driver. This was fixed per https://github.com/OSGeo/gdal/pull/6651.

sgillies commented 1 year ago

Enable the GML driver? Never! :laughing:

sgillies commented 1 year ago

CFLAGS are a mess. A bunch of bare arm64 and an -arch arm64-arch arm64. Not sure what the cause is.

      clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g arm64 arm64 -g -O2 -g -O2 -g -O2 -g -O2 -g -O2 -I/tmp/local/include -arch arm64-arch arm64 -I/private/var/folders/76/zy5ktkns50v6gt5g8r0sf6sc0000gn/T/cirrus-ci-build/venv/lib/python3.11/site-packages/numpy/core/include -I/tmp/local/include -I/private/var/folders/76/zy5ktkns50v6gt5g8r0sf6sc0000gn/T/cirrus-ci-build/venv/include -I/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c rasterio/_base.c -o build/temp.macosx-11.0-arm64-3.11/rasterio/_base.o -Wno-unused-parameter -Wno-unused-function
      clang: error: no such file or directory: 'arm64'
      clang: error: no such file or directory: 'arm64'
      clang: error: no such file or directory: 'arm64'
      clang: error: invalid arch name '-arch arm64-arch'
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
sgillies commented 1 year ago

I found and fixed a multibuild bug and we've got wheels :tada:

Thank you very much for the help @rouault. You're the best!

sgillies commented 1 year ago

Not quite ready to merge. My custom build breaks yum in the manylinux2014 image, so I need to avoid using get_modern_cmake for Linux builds.

sgillies commented 1 year ago

GDAL's new CMake build system tends to include more drivers than the old autotools system did. For example, it finds OpenEXR installed on GHA's x86_64 macos VM and decides to build the EXR driver. But OpenEXR comes from homebrew and isn't comptatible with the macos version we're building for. The old system didn't do this.

x86_64 mac builds are broken until I track down all the new drivers being included by CMake.

rouault commented 1 year ago

But OpenEXR comes from homebrew and isn't comptatible with the macos version we're building for. The old system didn't do this.

you could likely use CMake variables to restrict the search path, like https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html

sgillies commented 1 year ago

Homebrew openjpeg! I think that's the last conflict :crossed_fingers:

sgillies commented 1 year ago

Yep, that was it :upside_down_face: