Need to set LIBRARY_PATH for emacs-plus@28 with --with-native-comp

[poldy]

I ran brew install emacs-plus@28 --with-native-comp and it completed successfully. However, when I tried to run Emacs although the editor worked, there were loads of native compilation warnings.

The solution was to create a one-line ~/.emacs.d/early-init.el, with the following contents:

(setenv "LIBRARY_PATH" "/opt/homebrew/lib/gcc/11:/opt/homebrew/lib/gcc/11/gcc/aarch64-apple-darwin20/11.1.0")

I'd suggest adding this to the "gccemacs" section of the README.

This is on an M1 iMac, it works great now.

[jilen]

Start emacs from the Terminal instead of click the emacs icon works for me.

[kommen]

EDIT

Don't set LIBRARY_PATH as I originally suggested in this post, it is just fighting symptoms, not the root cause, see other comments below.

Original comment below ------------------------

I can confirm setting LIBRARY_PATH as @poldy suggests as an env in my bash/zsh config fixes native-comp errors. Without it set I got this (a bit redacted)

Native compiler error: (lambda (&optional arg6) (let ((f #'kill-emacs)) (funcall f arg6))), Compiling <home>/.emacs.d/eln-cache/28.0.50-b1d099bb/subr--trampoline-6b696c6c2d656d616373_kill_emacs_0.eln...
ld: library not found for -lgcc_s.1.1
libgccjit.so: error: error invoking gcc driver
Debugger entered--Lisp error: (native-ice "failed to compile" "<home>/emacs.d/eln-cache/28.0.50-b1d099bb/..." "error invoking gcc driver")

[d12frosted]

See this message. If you have to set LIBRARY_PATH that probably just means something is wrong with your environment. Works for me without any hacks like this. Though I must admit, never tested on M1.

[kommen]

@d12frosted thanks!

After reinstalling gcc and libgccjit (also from source) with no success, I poked around in my env:

It seems the issue was Spacemacs caching the env variables (including PATH) in the .spacemacs.env file. After adjusting the PATH there to include my newish homebrew M1 setup's /opt/homebrew/bin:/opt/homebrew/sbin: it looks I only get one native comp error regarding cl-lib, the other files now seem to compile fine.

[d12frosted]

it looks I only get one native comp error regarding cl-lib

Do you mean an error saying that it's obsolete/deprecated? 😄

the other files now seem to compile fine.

This sounds great.

[kommen]

Do you mean an error saying that it's obsolete/deprecated? 😄

Unfortunately no 😅. Still some native-comp error:

Compiling /opt/homebrew/Cellar/emacs-plus@28/28.0.50/share/emacs/28.0.50/lisp/emacs-lisp/cl-lib.el.gz...
uncompressing cl-lib.el.gz...
uncompressing cl-lib.el.gz...done
ld: library not found for -lgcc_s.1.1
libgccjit.so: error: error invoking gcc driver
/opt/homebrew/Cellar/emacs-plus@28/28.0.50/share/emacs/28.0.50/lisp/emacs-lisp/cl-lib.el.gz: Error: Internal native compiler error failed to compile

The others following immediately after it seemingly work:

Compiling /opt/homebrew/Cellar/emacs-plus@28/28.0.50/share/emacs/28.0.50/lisp/emacs-lisp/seq.el.gz...
uncompressing seq.el.gz...
uncompressing seq.el.gz...done
Compiling /opt/homebrew/Cellar/emacs-plus@28/28.0.50/share/emacs/28.0.50/lisp/emacs-lisp/gv.el.gz...
uncompressing gv.el.gz...
uncompressing gv.el.gz...done
...

[kommen]

I also got that last error fixed: it was because the compilation was kicked off by spacemacs before loading the .spacemacs.env file. This happens because when starting Emacs.app from the macOS Finder or Dock, the env vars are different then when launching from the command line (where I have my .zshrc). Launching Emacs from the commandline (executing /Applications/Emacs.app/Contents/MacOS/Emacs directly) so it inherits the env vars from there fixed it.

So I can confirm that these errors were entirely due to my environment Emacs runs in not being set up properly, especially that PATH is set correctly for Emacs when running with homebrew on an M1 Mac. I hope this helps other folks running into this.

[d12frosted]

People say that null pointers are billion dollars issue, but I'd rather say that it's PATH. I actually wonder if it's a good idea to patch info.plist file when emacs-plus is built so it includes PATH value during the build. That way it will automatically get the right (though possible out of date) PATH value without the need to resort to things like exec-path-from-shell or env from Spacemacs.

[elken]

People say that null pointers are billion dollars issue, but I'd rather say that it's PATH. I actually wonder if it's a good idea to patch info.plist file when emacs-plus is built so it includes PATH value during the build. That way it will automatically get the right (though possible out of date) PATH value without the need to resort to things like exec-path-from-shell or env from Spacemacs.

This is probably a better solution IMO than relying on users having correct environments. When built with native-comp, add the path during build so on startup the compiler can find libgccjit.

[eltone]

I had the same error after brew reinstall emacs-plus@28 --with-native-comp. I was able to fix it with:

brew unlink emacs-plus@28 && brew link emacs-plus@28

[d12frosted]

General advise - never use reinstall, especially with emacs-plus. See readme section for this.

Cheers, @.***

On Fri, Sep 3, 2021 at 12:08, Anthony Hobbs @.***> wrote:

I had the same error after brew reinstall @.*** --with-native-comp. I was able to fix it with:

brew unlink @.***

&&

brew link @.***

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[gwbrown]

In case anyone else finds this wants to change Info.plist fix manually after compiling, I found details on how to do so on this EmacsWiki page. Note that after editing Info.plist, you need to run /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -f <path_to_your_Emacs.app> or MacOS won't pick up on the fact that the plist has been updated and won't get the changed environment variables.

(That last bit had me pulling my hair out for a while)

[yuezhu]

On macOS 10.15 I was getting the same error while launching the Emacs application from the LaunchPad/Dock. It turned out the PATH environment variable isn't set correctly in this case. And I was able to work it around by adding the PATH into /usr/local/opt/emacs-plus@28/Emacs.app/Contents/Info.plist as follows:

# /usr/libexec/PlistBuddy -c "Add :LSEnvironment dict" /usr/local/opt/emacs-plus@28/Emacs.app/Contents/Info.plist
# /usr/libexec/PlistBuddy -c "Add :LSEnvironment:PATH string" /usr/local/opt/emacs-plus@28/Emacs.app/Contents/Info.plist
# /usr/libexec/PlistBuddy -c "Set :LSEnvironment:PATH $(echo "$PATH")" /usr/local/opt/emacs-plus@28/Emacs.app/Contents/Info.plist
# /usr/libexec/PlistBuddy -c "Print :LSEnvironment" /usr/local/opt/emacs-plus@28/Emacs.app/Contents/Info.plist
# touch /usr/local/opt/emacs-plus@28/Emacs.app

Make sure that the PATH environment variable is set correctly in your shell.

[DivineDominion]

On M1 Macs, the path to all Homebrew stuffs is different.

I'd suggest brew --prefix --installed emacs-plus@28 instead, so the calls become:

/usr/libexec/PlistBuddy -c "Add :LSEnvironment dict" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
/usr/libexec/PlistBuddy -c "Add :LSEnvironment:PATH string" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
/usr/libexec/PlistBuddy -c "Set :LSEnvironment:PATH $(echo "$PATH")" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
/usr/libexec/PlistBuddy -c "Print :LSEnvironment" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
touch `brew --prefix --installed emacs-plus@28`/Emacs.app

Without these fixes, I have to launch Emacs from the shell. Otherwise it just won't find libgccjit etc. during native compilation

Edit: Shorter

alias pbd=/usr/libexec/PlistBuddy
export emacsapp=`brew --prefix --installed emacs-plus@28`/Emacs.app
export infoplist="${emacsapp}/Contents/Info.plist"

pbd -c "Add :LSEnvironment dict" "${infoplist}"
pbd -c "Add :LSEnvironment:PATH string" "${infoplist}"
pbd -c "Set :LSEnvironment:PATH $(echo "$PATH")" "${infoplist}"
pbd -c "Print :LSEnvironment" "${infoplist}"
touch ${emacsapp}

[jimeh]

Not sure if it's all that helpful, but I mess about with the LIBRARY_PATH environment variable with my custom build script that attempts to produce a fully self-contained Emacs.app. Because it copies gcc and libgccjit stuff into the .app bundle, I need to point LIBRARY_PATH at the specific directories within the bundle.

Potentially emacs-plus can do something similar, by simply ensuring it's pointing at the correct brew paths.

In my case, I add some custom elisp to Emacs.app/Contents/Resources/lisp/site-start.el, which if it exists, is loaded before both init.el and early-init.el from user config.

For reference, this is what the elisp my build-script produces looks like:

;; Set LIBRARY_PATH to point at bundled GCC and Xcode Command Line Tools to
;; ensure native-comp works.
(when (and (eq system-type 'darwin)
           (string-match-p "\.app\/Contents\/MacOS\/?$"
                           invocation-directory))
  (let* ((library-path-env (getenv "LIBRARY_PATH"))
         (devtools-dir
          "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib")
         (gcc-dir (expand-file-name
                   "lib/lib/gcc/11"
                   invocation-directory))
         (darwin-dir (expand-file-name
                      "lib/lib/gcc/11/gcc/x86_64-apple-darwin19/11_2_0"
                      invocation-directory))
         (lib-paths (list)))

    (if library-path-env
        (push library-path-env lib-paths))
    (if (file-directory-p devtools-dir)
        (push devtools-dir lib-paths))
    (push darwin-dir lib-paths)
    (push gcc-dir lib-paths)

    (setenv "LIBRARY_PATH" (mapconcat 'identity lib-paths ":"))))

The above obviously won't work as is in an emacs-plus Emacs.app bundle, but it could potentially be adjusted to detect/find or just hard-code the relevant homebrew paths for GCC.

[jsmestad]

On M1 Macs, the path to all Homebrew stuffs is different.

I'd suggest brew --prefix --installed emacs-plus@28 instead, so the calls become:

/usr/libexec/PlistBuddy -c "Add :LSEnvironment dict" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
/usr/libexec/PlistBuddy -c "Add :LSEnvironment:PATH string" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
/usr/libexec/PlistBuddy -c "Set :LSEnvironment:PATH $(echo "$PATH")" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
/usr/libexec/PlistBuddy -c "Print :LSEnvironment" `brew --prefix --installed emacs-plus@28`/Emacs.app/Contents/Info.plist
touch `brew --prefix --installed emacs-plus@28`/Emacs.app

Without these fixes, I have to launch Emacs from the shell. Otherwise it just won't find libgccjit etc. during native compilation

Edit: Shorter

alias pbd=/usr/libexec/PlistBuddy
export emacsapp=`brew --prefix --installed emacs-plus@28`/Emacs.app
export infoplist="${emacsapp}/Contents/Info.plist"

pbd -c "Add :LSEnvironment dict" "${infoplist}"
pbd -c "Add :LSEnvironment:PATH string" "${infoplist}"
pbd -c "Set :LSEnvironment:PATH $(echo "$PATH")" "${infoplist}"
pbd -c "Print :LSEnvironment" "${infoplist}"
touch ${emacsapp}

This worked great for me. Thank you 🙏

[tilgovi]

This appears to have worked for me, as well, on an Intel mac.

[jsmestad]

@d12frosted would it make sense to add this to the main script? I have to manually apply this fix to every install on my M1 mac or launching from macOS launcher has not found errors

[d12frosted]

@jsmestad in general, it's a bad idea to modify LIBRARY_PATH and other things like this. This simply means that something is wrong in the environment. See https://github.com/d12frosted/homebrew-emacs-plus/issues/378#issuecomment-903160417.

The only place where it's valid (IMHO) is Emacs.app own Info.plist file. I have an idea of injecting several environment variables into Info.plist file to avoid plethora of issues people run into when running Emacs.app due to how macOS works. This includes, but is not limited to PATH and LIBRARY_PATH. This will not affect running emacs from terminal as it will simply inherit interactive session. I just need to find some time to play around with this idea. Maybe next week? 🤷

WDYT @jsmestad ?

[jsmestad]

Yeah I agree. Interested to see how you approach it!

The macOS launcher issue has been hurting folks adopting Emacs, at least at my office, because they think it's too difficult/buggy to get native-comp working when in-fact this has nothing to do with native-comp. 🫤

[d12frosted]

@jsmestad started toying around with this idea in https://github.com/d12frosted/homebrew-emacs-plus/pull/453. It was not obvious how to get real user PATH instead of one from brew env.

For now doing it for emacs-plus@29 only.

Right now the build is red because GitHub packages degraded according to https://www.githubstatus.com

In any case, I would love someone to test it 🙏

[d12frosted]

Ok, it's green now. So I would appreciate some testing. In case no one reports issues till Wednesday, I will merge it :)

[aaronjensen]

Worked for me!

[tilgovi]

emacs-plus@29 --with-native-comp working fine for me, too, with no plist hacking.

[d12frosted]

Merged #453, which should ease usage of native compilation feature. Please let me know if you still encounter any issues.

[jsmestad]

@d12frosted sorry for the delay. The patch works great. Thanks!

[d12frosted]

@jsmestad glad to hear :)

[sainathadapa]

Just in case it helps anyone in the future, after quite a bit of trial and error, the following steps worked:

Uninstalled gcc and libgccjit

brew uninstall --ignore-dependencies libgccjit

Uninstalled any existing emacs-plus installations

brew uninstall emacs-plus@29

Installed libgccjit from source

brew install --build-from-source libgccjit

Unlinked and relinked libgccjit

brew unlink libgccjit && brew link libgccjit

Installed emacs-plus

brew install emacs-plus@29 --with-native-comp --with-imagemagick --with-no-frame-refocus

Added the following statement to .emacs.d/init.el

(setenv "LIBRARY_PATH"
    (mapconcat 'identity
     '(
       "/usr/local/Cellar/gcc/13.1.0/lib/gcc/13/"
       "/usr/local/Cellar/libgccjit/13.1.0/lib/gcc/13/"
       "/usr/local/Cellar/gcc/13.1.0/lib/gcc/13/gcc/x86_64-apple-darwin22/13/"
       "/opt/homebrew/opt/gcc/lib/gcc/13"
       "/opt/homebrew/opt/libgccjit/lib/gcc/13"
       "/opt/homebrew/opt/gcc/lib/gcc/13/gcc/aarch64-apple-darwin22/13")
         ":"))

[vintonc]

I had this issue today after "sudo port upgrade outdated" on Monterey MacOS 12.7.5 on a 2015 MBP. I tried everything here and anywhere else I could find. What ended up working for me was "rm -rf /Library/Developer/CommandLineTools" to delete the command line tools directory and then reinstalling them "xcode-select --install" this pops up a dialog asking to reinstall. It tool almost 20 minutes to reinstall