[mac][wx] build failure: wxWidgets 3.1.0 incompatible with macOS 10.15

[brevilo]

Describe the bug

wxWidgets-3.1.0/src/osx/webview_webkit.mm:35:10: fatal error: 'WebKit/HIWebView.h' file not found
#include <WebKit/HIWebView.h>
         ^~~~~~~~~~~~~~~~~~~~
wxWidgets-3.1.0/src/osx/webview_webkit.mm:35:10: note: did not find header 'HIWebView.h' in framework 'WebKit' (loaded from '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks')

Steps To Reproduce

1. Run setupForBOINC.sh (or just buildWxMac.sh)

System Information

• OS: macOS 10.15 (Xcode 11.3.1)
• BOINC Version: master

This again raises the question how this could escape integration testing. Which macOS platforms does Travis test on?

[brevilo]

I can confirm that wxWidgets 3.1.3 fixes the issue above. However, I'm not going to provide a PR against dependencyNames.sh (only) since buildWxMac.sh contains a number of patches and workarounds for 3.1.0 that I can't evaluate myself.

[brevilo]

Turns out that an upgrade to wxWidgets 3.1.3 isn't as straight-forward as I'd hoped and building BOINC itself now fails like this (after modifying the Xcode project for 3.1.3):

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSApplication.h:61:30: error: expected unqualified-id
static const NSAppKitVersion NSAppKitVersionNumber10_10 = 1343;
                             ^
In file included from boinc/clientgui/mac/MacAccessiblity.mm:20:
In file included from boinc/clientgui/mac/MacGUI.pch:79:
In file included from ../../wxWidgets-3.1.3/include/wx/wxprec.h:12:
In file included from ../../wxWidgets-3.1.3/include/wx/defs.h:45:
../../wxWidgets-3.1.3/include/wx/platform.h:58:48: note: expanded from macro 'NSAppKitVersionNumber10_10'
#            define NSAppKitVersionNumber10_10 1343
                                               ^

A related error crops up for NSAppKitVersionNumber10_10 as well.

Update:

Ok, the reason for the above is that BOINC defines MAC_OS_X_VERSION_MAX_ALLOWED as 1070 but wxWidgets (as of 3.1.1) requires at least 101300 for this to work (tested).

[brevilo]

Oh for ****'s sake!

Update: Background is explained here. Looking into it...

Update 2: Since wxWidgets changed their API in a backwards-incompatible way, without even mentioning that in their release notes, I'm done here.

[brianclinkenbeard]

@brevilo what can I do to fix building WxWidgets? Just modify the scripts to use 3.1.3?

I have to confess, the current build process for macOS seems to be fairly convoluted. It seems that the scripts work well to automate the build process but also lead a lack of robustness that results in issues like these.

For example, couldn't dependencies simply be pulled in with homebrew and software like CMake or Meson could be used to build and pull in any additional externals to the project? This seems to be the most common way I see with most modern cross-platform projects. Or do these scripts remedy some other issue I'm unaware of? Forgive me if I'm missing something.

[CharlieFenton]

The build scripts install multiple patches to fix problems specific to that version of wxWidgets (currently we use 3.1.0) and some of the other dependent libraries. So it is essential that the build scripts be used. Updating to a newer version of wxWidgets usually introduces new bugs which need to be analyzed and new patches created for them.

Currently, wxWidgets 3.1.0 doesn't build under OS 10.15 / Xcode 11, apparently because the headers HIWebView.h and CarbonUtils.h have been removed from the WebKit framework in the Mac OSX 10.15.sdk in Xcode 11. They do build properly with Mac OSX 10.14.sdk, which is provided in Xcode 10. It may be possible to fix this by adding more patches to the buildWxMac.sh script.

It is possible to automate the downloading of the dependent library sources. This has been implemented for the Travis CI builds, but not yet for individual builds at 3rdParty/buildMacDependencies.sh.

[CharlieFenton]

Note also that updating to a newer version of any dependent library requires extensive testing of the code produced, since subtle bugs may be introduces that only show up under specific conditions at run time.

[brevilo]

Guys, I was hoping to have described what's needed to upgrade to wx 3.1.3 above after I did all the detective work over almost a full day. Let me know what's still unclear.

[brianclinkenbeard]

@CharlieFenton Couldn't WxWidgets just be brought in through an externals folder? I know that CMake has the capability to automatically apply patches to externals. It looks like autoconf is used for Linux builds, I wonder if it has similar capabilities.

[brianclinkenbeard]

@brevilo Updating to WxWidgets 3.1.3 makes the build work, then commenting out the patches in buildWxMac.sh and updating dependencyNames.sh allows me to successfully run setupForBOINC.sh. When I try to build the project however, I get errors having to do with Cocoa. I will see if I can get the patches working on 3.1.3 and if that fixes it.

[CharlieFenton]

I have created PR ##3483 which fixes this for wxWidgets 3.1.0. The change was fairly trivial.

Note that building without compiler errors does not guarantee that linking with a different version of wxWidgets won't result in new run-time problems.

I have purposely hard-coded the version of each dependent library to force a build failure if someone attempts a build with a different library version. That helps ensure that every time someone makes a build of BOINC for the Mac using that Xcode project, the resulting runtime code will be functionally identical. This is important for multiple reasons, such as facilitating the debugging of problem reports from the field.

[CharlieFenton]

It looks like autoconf is used for Linux builds, I wonder if it has similar capabilities.

As I wrote in my comments to PR #3471:

If you use configure to create the config.h file, the generated code may not work on older versions of OS X, and may not work at all (as implied by PR #3471.) The Xcode build uses a fixed version of config.h at clientgui/mac/config.h, and so should the make file.

As I understand it, the config / make approach was developed on a model where each user would build most software themselves, on their target computer, from the source files. This starts with running configure on that target computer, since configure determines the build options by testing what resources are available on that peculiar machine and optimizes the build to run on that particular machine.

But those resources may not be available on versions of OS X older than the one on the build machine. This is one of the reasons I created a special config.h for Mac builds at clientgui/mac/config.h. All that is needed to use this is to put that path ahead of others in the search path, or to put a symbolic link to it where the make file expects to find config.h, and of course just don't run configure.

(FWIW, Apple bases their limited approach to backward compatibility on a very different model, using availability macros in its headers.)

[CharlieFenton]

Note that building without compiler errors does not guarantee that linking with a different version of wxWidgets won't result in new run-time problems.

... not just on the current version of OS X, but possibly only under older versions of OS X. A change in dependent library requires thorough alpha testing with older OS X systems as well as the current one. Over the many years that I was the primary (often only) Mac programmer on the BOINC project, I have strived to maintain backward compatibility to the maximum extent possible. Currently, BOINC supports all versions of OSX back to OS 10.7.0. This is especially valuable because many BOINC volunteers dedicate their old computers (not just Macs) to running BOINC when they buy a newer computer, and newer operating systems those older machines may not be able to run newer OS's.

[CharlieFenton]

@brianclinkenbeard and @brevilo: I realize my comments all may come across as very negative, so I want to emphasize that I am very glad that you are getting involved with BOINC development for the Mac. Since my (attempted) retirement form software engineering, I have been concerned that Mac support for BOINC has languished.

Please understand that my extreme caution regarding changes stems from my 40+ years of experience as a software engineer and my many years supporting the original stand-alone SETI@home and then BOINC on the Mac. My comments are intended to help others avoid some of th pitfalls I have encountered, and are intended to be helpful and supportive.

My caution has been heightened by the fact that we still have never had a release of BOINC which can be installed under OS 10.15 Catalina. Apple released OS 10.15.0 almost 5 months ago on October 7, 2018, OS 10.15.1 on October 29, 2019, 10.15.2 on December 10, 2020 and 10.15.3 on January 28, 2020.

This is because:

[1] BOINC 7.16.3 was released without first doing Apple's "notarization" step, without which Catalina refuses to install applications downloaded from the Internet (though expert users have a way to work around this restriction.) This was supposed to be corrected in BOINC version 7.16.4, but ...

[2] BOINC 7.16.4 included every change made to GIT master up to that time without adequate testing, instead of cherry-picking only bug fixes as dictated by the Client Release Process description in the BOINC Governance Documents. This introduced show-stopper bugs which required the withdrawal of that build, and no new build has been done since. As a result, no version 7.16.4 was ever built for the Mac.

[3] Apple added significant restrictions to what screensavers are permitted to do in Catalina, so the bOINC screensavers no longer work under OS 10.15. I made major changes / additions to the screensaver code to work around these new restrictions. These were to be included in the next release of 7.16.x.

So I am very wary of additional significant changes until we have a successful Mac build for Catalina.

[RichardHaselgrove]

[2] BOINC 7.16.4 included every change made to GIT master up to that time without adequate testing, instead of cherry-picking only bug fixes as dictated by the Client Release Process description in the BOINC Governance Documents. This introduced show-stopper bugs which required the withdrawal of that build, and no new build has been done since. As a result, no version 7.16.4 was ever built for the Mac.

While this conversation has been going on, @davidpanderson has further updated the 7.16 release branch, with what looks like a further bulk (unselective) import from master - commit ef151e8de664fd8ad8aa6bd83dbb56633fae5d0a (Feb 21). The release branch was tagged to 7.16.5 on Feb 25, and 64-bit Windows builds were placed in https://boinc.berkeley.edu/dl/ the same day.

I have not yet tested the Windows installer, but I have made a private build from a clean download of v7.16.5, and so far that seems to be running without problems.

[brevilo]

So I am very wary of additional significant changes until we have a successful Mac build for Catalina.

Ok, I was just about to ask if wouldn't make more sense to get wx 3.1.3 to work instead of fixing the outdated 3.1.0 build. But your statement makes sense in the context of the 7.16 "situation". I'll try and get your PR reviewed/merged.