QGIS 3.28 requires GEOS 3.10, packager is using 3.9

[justinbb]

QGIS 3.28 builds do not work on Mac (attempts to use the "Fix Geometries" command result in a crash) because QGIS 3.28 requires GEOS 3.10 as a result of https://github.com/qgis/QGIS/pull/50070. The Mac packager has not been updated. (How is this possible, is there no coordination between the different parts of the QGIS project?) See https://github.com/qgis/QGIS/issues/50888 for the fallout.

[jctull]

Unfortunately, it is starting to appear as if the Mac packager is currently abandoned or unmanaged. The feature parity is greatly diminished for the Mac with the lack of current support packages (gdal, geos, etc.). This is feeling like familiar territory from the past, unfortunately.

[justinbb]

Indeed, now that I've looked around the situation is completely dire. However, others have been moving on: see https://github.com/qgis/QGIS/issues/46299 where up-to-date Mac builds are being discussed. They can be obtained using either MacPorts or Homebrew. The future is promising after all. What is missing is some way of getting one of those versions signed, notarized and packaged – hopefully without the complexity and unmaintainability of the current packager.

[jctull]

Indeed, now that I've looked around the situation is completely dire. However, others have been moving on: see qgis/QGIS#46299 where up-to-date Mac builds are being discussed. They can be obtained using either MacPorts or Homebrew. The future is promising after all. What is missing is some way of getting one of those versions signed, notarized and packaged – hopefully without the complexity and unmaintainability of the current packager.

It looks like those projects have the same issue with lagging dependencies. I don't believe either of them has moved to current GDAL, though that may still be due to underlying bugs in qgis. I did not read the entire thread. I installed a qgis docker container for some critical geometry repairs that were crashing under qgis-mac, which is less than ideal, but workable.

[jctull]

I did some minor testing with QGIS installed with condo-forge: https://github.com/conda-forge/qgis-feedstock

The underlying libraries are more current than what is available in the official Mac package at this time. A test of fix geometries resulted in correct output/no crash.

[agiudiceandrea]

@justinbb @jctull, please see https://github.com/qgis/QGIS/pull/52360, which allegedly fixes the issue reported at https://github.com/qgis/QGIS/issues/50888.

[justinbb]

@agiudiceandrea Thank you: the immediate fire has been put out. The basic problem with the Mac packager is still present, however. It is out of date for a large number of QGIS dependencies, it requires separate maintenance instead of updating automatically along with the Linux build, yet it is apparently unmaintainable and has been abandoned by its creators. On top of all that, it's an incomplete solution to the Mac packaging problem as the resulting Mac app bundle does not conform to bundling rules and, despite being signed by an Apple Developer ID, is not (cannot be?) notarized (therefore the system still generates a warning when it is launched). Do we retitle this bug for the wider problem, or create a new issue?

[justinbb]

Developing further on the problems:

• The reason this bug (and several others) happened in the first place is because the QGIS-Mac-Packager project does not assemble the dependencies the same way the other QGIS platforms do. It has its own custom-built and extremely manual system, which is found in https://github.com/qgis/QGIS-Mac-Packager/tree/master/qgis_deps. Updating that vast collection requires editing 106 separate configuration files located in 106 separate directories. Peter Petrik of Lutra Consulting seems to have been the original author of the system and certainly the principal maintainer until September 2021, when he abruptly stopped. No dependent libraries have been updated since.
• The relationship between the qgis_deps system and the rest of the QGIS-Mac-Packager is a loose coupling. I think it would be possible to keep the rest of the Packager while jettisoning the unmaintainable dependency collection. Ideally the dependencies would be assembled in the same way as is done for the regular Linux or Windows builds, and kept up to date at the same time and in the same way as for Linux and Windows. If the dependency libraries were pulled from (say) MacPorts or conda, they would minimally need to have their linker "install path" patched using Apple's install_name_tool. However, there may be other configuration issues (paths to configuration files or data? compilation-time configuration options? addition of non-open-source 3rd-party libraries?) for some packages, a problem which is no doubt already solved for Windows and Linux.
• The Packager still needs to be improved to make a better, compliant app bundle which would allow the final result to be notarized by Apple so it will run without warnings. (It is already being signed with an Apple Developer ID.) I can help with this part of the problem as I have at least the basics of Apple culture necessary to understand bundling issues. There is helpful documentation for this specific problem (Apple-bundling of open-source projects) in this Apple developer document: https://developer.apple.com/documentation/xcode/embedding-nonstandard-code-structures-in-a-bundle
• Despite the persistent problems packaging QGIS for the Mac, there is no fundamental issue blocking anything. The Mac is a first-class Unix development environment and is not missing any capability. It does, however, impose extra requirements for security and for user convenience, requirements which are culturally foreign to the open-source community in general. Yet Windows presents a not dissimilar situation, while also being a non-Unix platform… really, the only difference is the lack of experience with Mac development.

As I said, I can help with the Apple parts. The problem is I have practically no background in the QGIS development culture and all I know about previous Mac packaging efforts has been gleaned from bug reports and README files. I am a former developer of Mac software now trying to be a user of Mac QGIS. The latter being a very difficult proposition these days, I am trying to do what I can, however clumsily, to get the Mac packaging issue taken into serious consideration by the people who have some control over it.

[rouault]

@justinbb FYI this issue has been raised to the QGIS Project Steering Committee in email thread https://lists.osgeo.org/pipermail/qgis-psc/2023-March/thread.html#10011

[justinbb]

More history worth recalling. When the current QGIS-Mac-Packager was developed in 2020, it was considered an advantage to get away from existing package managers (they were using Homebrew at the time) and do it all by hand. See https://github.com/qgis/QGIS-Enhancement-Proposals/issues/177

Obviously I do not agree with this choice, and I think experience has pretty much confirmed that it has proved unworkable. (I note also that the OSGeo4A project on which the dreaded qgis_deps architecture was based is no longer in use; its authors at opengis.ch state "maintaining this is an immense effort we wanted to avoid" and they now use vcpkg.)

QGIS for Mac can't be a huge burden: it needs a high degree of automation and tight alignment with the Linux and Windows builds, especially in the management of dependencies.

[tsmcgrath]

@justinbb if you need any program management support on this issue let me know. Unfortunately my engineering skills have deteriorated and I won't be much help on the engineering side. But, it sounds like you're experience covers that. Anyway, let me know if I can help.

[agiudiceandrea]

Please see https://github.com/qgis/QGIS-Enhancement-Proposals/issues/270.

[justinbb]

@tsmcgrath My software skills have also deteriorated. For other reasons given above, I have not volunteered for more than helping decode things – writing up the analysis of the problem on this page, and possibly some assistance decoding strange Apple ways of doing things, if the need arises. As noted by @agiudiceandrea, the Mac packaging problem is now in the capable hands of @m-kuhn (who is an experienced QGIS developer) and he has funding. We just need to be patient. In the meantime, it's possible to work using QGIS from conda (or MacPorts).

[jctull]

I was using conda on my M2 MacBook Pro, but found macports to be a more stable and unified option. If you are deeply invested in homebrew, conda may be your best option until things sort out on the official binary.

[m-kuhn]

I was using conda on my M2 MacBook Pro, but found macports to be a more stable and unified option. If you are deeply invested in homebrew, conda may be your best option until things sort out on the official binary.

What are the shortcomings on conda side exactly?

[jctull]

I was using conda on my M2 MacBook Pro, but found macports to be a more stable and unified option. If you are deeply invested in homebrew, conda may be your best option until things sort out on the official binary.

What are the shortcomings on conda side exactly?

It's been a few months since I was using it, but I believe there were issues with ssl or certificates. Additionally, installing requirements for plugins was non-intuitive. Overall, you are basically managing an env, and that layering creates more complexity and potential points of failure.