search

caldwell / build-emacs

Build scripts for www.emacsformacosx.com
http://www.emacsformacosx.com/about
GNU General Public License v3.0
364 stars 61 forks source link

Build on Big Sur #103

Closed dgreatwood closed 2 years ago

dgreatwood commented 3 years ago

I had previously built emacs 26.2 on my laptop under Catalina. Now having upgraded to macos Big Sur, I'm trying to build emacs 27.1.

After proceeding a good distance in the build, there are a number of warnings of this form: `Warning: Building p11-kit from source as the bottle needs:

After 7 of these warnings, we get: Error: The following formulae cannot be installed from bottles and must be built from source. guile, gettext, libidn2, p11-kit, openssl@1.1, nghttp2 and unbound brew failed And the build stops with the error. The modules listed are the same ones that previously generated the warnings.

All the modules listed are installed per brew and are up to date.

Suggestions?

dgreatwood commented 3 years ago

Further, I noticed this Homebrew discussion: https://github.com/Homebrew/discussions/discussions/272 Specifically, @Rylan12 (homebrew maintainer) wrote that:

I think the change that has been made was only for cases when HOMEBREW_REPOSITORY == HOMEBREW_PREFIX which appears to be true in your case. If, instead, you can set up the HOMEBREW_REPOSITORY to be in e.g. /Users/jaraco/.local/homebrew/Homebrew that might allow this particular bottle to continue to work.

In the emacs build above, where the emacs build system is using its own homebrew cellar and repository, it is also the case that HOMEBREW_REPOSITORY == HOMEBREW_PREFIX, against this apparent advice of @Rylan12. Does this need to be changed for Big Sur compile / compile using latest homebrew? E.g. HOMEBREW_REPOSITORY could be set to .../build-emacs-master/brew/Homebrew

Thoughts?

dgreatwood commented 3 years ago

OK, so there is a workaround, which is to add "--no-brew" to the build command, invoking thus: ./build-emacs-from-tar -v --no-brew -j 8 emacs-XX.YY.tar.bz2 release (replacing XX.YY with emacs version) I think this stops the build downloading and building its own brew tools, so it is likely more dependent on whatever brew tools are already installed on the Mac.

xxyzz commented 3 years ago

Why not install Homebrew normally?

dgreatwood commented 3 years ago

Why not install Homebrew normally?

I believe that is what my "workaround" does, effectively. The emacs build scripts by default download and use their own copy of Homebrew; the scripts decide how the new copy of brew is setup. By using "--no-brew", the emacs build scripts instead use the brew that's installed on my machine already, which (I guess) has been installed "normally".

Or did you have something else in mind in terms of installing brew "normally"?

xxyzz commented 3 years ago

I mean why ensure-brew doesn't use the recommend command to install Homebrew(/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)")?

dgreatwood commented 3 years ago

I mean why ensure-brew doesn't use the recommend command to install Homebrew(/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)")?

Well, @caldwell is the author of ensure-brew, maybe he can comment?

Again, perhaps it has to do with making a "private" copy of brew, independent of the machine's own brew configuration.

caldwell commented 3 years ago

ensure-brew was not meant to install brew machine wide. Installing stuff machine wide is always a bad idea on a build box. That script was purposefully designed to make a local brew that didn't taint other builds that may happen on the box. It was also useful because on different OSes I would peg brew at different git commits (because they aggressively drop support of older OSes that I still wanted to support). Unfortunately brew became more and more untenable as they stopped supporting anything but a default system-wide install. (Even trying to compile from source instead of using a bottle yells at you now!). To be fair, I was basically abusing brew to get this functionality—I was trying to make it Nix and it just isn't.

The latest version of the build scripts (which I just now remembered I hadn't pushed to github) rips out ensure-brew entirely and the dependencies are now built from scratch (think of it as kind of a brew-lite). The upside to this is that no one can pull functionality out from under it any more, so it shouldn't randomly break as often. The downside is that all the versioning is handled explicitly, and no one is out there making sure things just build. I already had to patch one codebase to make it work on M1.

dgreatwood commented 3 years ago

@caldwell, thanks for updating the build scripts on github. Testing this, it runs through on my MAC machines except that the build script can't find the guile package. Since I already have guile 3.0.5 installed under brew, I tried: export PKG_CONFIG_PATH="/usr/local/opt/guile/lib/pkgconfig" and then rerunning build. Build then completes.

There does seem to be an issue still with M1 MAC. Specifically:

Note, on "27-2 built by me on M1 MAC crashes on M1 Mac during startup", the Apple crash reports:

Exception Type: EXC_BAD_ACCESS (Code Signature Invalid) Exception Codes: 0x0000000000000032, 0x00000001009a0000 Exception Note: EXC_CORPSE_NOTIFY Termination Reason: Namespace CODESIGNING, Code 0x2

Would be cool to address the guile issue and the apparent M1 issue(s).

Thanks for maintaining emacsformacosx.com! -D

dgreatwood commented 3 years ago

OK, I managed to get the build-it-myself Emacs to work on M1 MAC :-) Leaving some notes here in case of interest.

Since Bug Sur macOS requires app signing on M1, we have to code sign the app. The emacsformacosx build script combine-and-package has an option for this, -- sign IDENTITY.

First, we create a signing certificate using Xcode. Note, you don't have to be a member of the Apple development program to do this. Reference: https://help.apple.com/xcode/mac/current/#/dev154b28f09?sub=devd6ad10e16, "Create, export, and delete signing certificates". In Xcode, go to Preferences => Accounts. If you don't have an account shown, add one by clicking on the button bottom left and logging in using your regular Apple ID. Select a Team in the window, and click on Manage Certificates (if no team is showing, but you are logged in using your Apple ID, quit Xcode and restart Xcode - the team should show up on the second attempt). In the signing certificates window, press + bottom left and add a certificate. Then right-click on the certificate and Export it to a file, e.g. in your Downloads folder on your Mac.

Second, we have to add the signing cert to a keychain on your mac. Run the Keychain Access app on your Mac. Select the Login keychain (on the left), and select the Certificates window (top right). Now drag the certificate file you exported from Finder into the Certificates window of the Keychain Access app. You'll see the new certificate in the list. It will typically have a name of the form "Apple Development: YOUR NAME (YOUR ID)". Right-click on the newly installed certificate and do "Get Info". In the Info window, highlight the common name and copy it (e.g. triple-click the name and CMD-C to copy to clipboard).

Third, we need to install an intermediate signing certificate from Apple. Click on this link to download the cert from Apple: https://www.apple.com/certificateauthority/AppleWWDRCAG3.cer In Finder (Downloads) double-click the certificate file to install it.

Then we go through the usual Emacs build process, using fetch-emacs-from-ftp and build-emacs-from-tar (see https://github.com/caldwell/build-emacs). Once we reach the combine-and-package stage, we include the signing identity thus: ./combine-and-package -v --sign "COMMON NAME" Emacs-27.2-11.2-arm64.tar.bz2 replacing COMMON NAME with the certificate common name copied from the Keychain Access app.

@caldwell maybe some version of these notes should be added to the https://github.com/caldwell/build-emacs instructions?

Thanks!

thynus commented 2 years ago

Thank you for writing that note on Codesigning. I discovered my x86 MBP did not have a valid cert.