alyssais
commented
3 years ago cc @adisbladis
Open sheijk opened 3 years ago
alyssais
commented
3 years ago cc @adisbladis
jcf
commented
9 months ago I think I've tracked down the source of the problem, which affects versions of Emacs built with Home Manager.
The trivial runCommand builder has its name overridden to append with-packages, but the same modification is not made to emacs, which substitute depends on via prog. With such a change, the call to substitute should behave as expected.
substitute ${./wrapper.sh} $out/Applications/Emacs.app/Contents/MacOS/Emacs \
--subst-var-by bash ${emacs.stdenv.shell} \
--subst-var-by wrapperSiteLisp "$deps/share/emacs/site-lisp" \
--subst-var-by wrapperSiteLispNative "$deps/share/emacs/native-lisp" \
--subst-var-by prog "$emacs/Applications/Emacs.app/Contents/MacOS/Emacs"
chmod +x $out/Applications/Emacs.app/Contents/MacOS/Emacs
wrapProgramBinary $out/Applications/Emacs.app/Contents/MacOS/EmacsIf you have Home Manager setup, you can reproduce the issue with the following configuration:
{
programs.emacs = {
enable = true;
package = pkgs.emacs-unstable-pgtk;
extraPackages = epkgs: [epkgs.vterm];
overrides = final: prev: {
# `emacs-28` patches are compatible with `emacs-29`.
#
# Where a compatible path exists, there is a symlink upstream to keep
# things clean, but GitHub doesn't follow symlinks to generate the
# responses we need (instead GitHub returns the target of the symlink).
patches =
(prev.patches or [])
++ [
# Fix OS window role (needed for window managers like yabai)
(pkgs.fetchpatch {
url = "https://raw.githubusercontent.com/d12frosted/homebrew-emacs-plus/master/patches/emacs-28/fix-window-role.patch";
sha256 = "0c41rgpi19vr9ai740g09lka3nkjk48ppqyqdnncjrkfgvm2710z";
})
# Use poll instead of select to get file descriptors
(pkgs.fetchpatch {
url = "https://raw.githubusercontent.com/d12frosted/homebrew-emacs-plus/master/patches/emacs-29/poll.patch";
sha256 = "0j26n6yma4n5wh4klikza6bjnzrmz6zihgcsdx36pn3vbfnaqbh5";
})
# Enable rounded window with no decoration
(pkgs.fetchpatch {
url = "https://raw.githubusercontent.com/d12frosted/homebrew-emacs-plus/master/patches/emacs-29/round-undecorated-frame.patch";
sha256 = "0x187xvjakm2730d1wcqbz2sny07238mabh5d97fah4qal7zhlbl";
})
# Make Emacs aware of OS-level light/dark mode
(pkgs.fetchpatch {
url = "https://raw.githubusercontent.com/d12frosted/homebrew-emacs-plus/master/patches/emacs-28/system-appearance.patch";
sha256 = "14ndp2fqqc95s70fwhpxq58y8qqj4gzvvffp77snm2xk76c1bvnn";
})
];
};
};
}Mentioning Emacs maintainers for input before I open a pull request to fix this: @AndersonTorres @adisbladis @atemu @jwiegley @lovek323 @matthewbauer
I'm working on a reproducible example of this issue to ease testing, and I think I know what's causing the problem — while the runCommand call has -with-packages added, the prog substitution doesn't.
Would a pull request that applies the with-packages rename in both places solve this problem? If so, I'll work on something along those lines this weekend.
Update: I've tested my hypothesis locally, and it doesn't fix the problem. Executing the original Emacs derivation (sans with-packages suffix) is the desired behaviour. If, instead, you execute the with-packages version, Emacs never finishes launching.
Back to the drawing board!
binarin
commented
3 days ago The other thing appears to be affected by this - native compilation, which also uses invocation-name and invocation-directory - https://github.com/emacs-mirror/emacs/blob/8903106bb783c2825233c149b6799960aacdea57/lisp/emacs-lisp/comp-run.el#L302
And when emacs e.g. tries to automatically native-compile user's init.el, it can cause really funny side-effects during compilation, like (use-package :ensure t) installing package to user's .emacs.d directory (despite them already being installed emacsWithPackages)
The change introduced in https://github.com/NixOS/nixpkgs/pull/106486 (commit https://github.com/NixOS/nixpkgs/pull/106486/commits/23d4bfb6661ca57a9e331a2cf4184232d38ac38b) says that load-path sanitization is only for convenience when hacking on emacs itself or it's nixpkgs packaging.
Now that it's affecting 2 core features of emacs itself - emacs-restart and native compilation, maybe the convenience of emacs hacker/packager should be given a lower priority, with load-path sanitization removed or hidden behind an option. CC @alyssais
The last test case of the following shell script shows that without sanitization (as it's inhibited by -Q) nested emacs processes would be working in the way emacs itself expects them to work.
#!/usr/bin/env bash
nix build --impure --expr '
{ pkgs ? import <nixpkgs> {} }:
let
myEmacs = pkgs.emacs-nox;
emacsWithPackages = (pkgs.emacsPackagesFor myEmacs).emacsWithPackages;
in
emacsWithPackages (epkgs:
(with epkgs.elpaPackages; [ undo-tree ]))
'
E=./result/bin/emacs
cat <<'EOF' > test.el
(message "load-path has %s items" (length load-path))
(message "UNDO_TREE is installed?: %s" (package-installed-p (quote undo-tree)))
EOF
cat <<'EOF' > nested.el
(with-temp-buffer
(call-process
(expand-file-name invocation-name invocation-directory)
nil
(current-buffer)
nil
"-Q" ;; doesn't matter whether it's "-Q" or "-q"
"--batch"
"--load"
"test.el")
(message "NESTED:\n%s" (buffer-string)))
EOF
echo "======== Running directly: -q"
$E -q --batch --load test.el
echo -e "\n======== Running directly: -Q"
$E -Q --batch --load test.el
echo -e "\n======== Running as native compilation does it"
$E -q --batch --load nested.el
echo -e "\n======== Running as native compilation does it, but outer emacs with -Q - doesn't sanitize load-path"
$E -Q --batch --load nested.el
Describe the bug
When wrapping Emacs using emacsPackagesFor the invocation-directory will point to the wrong Emacs package. This means restarting Emacs from inside Emacs will result in a binary which doesn't see the additional packages to be started and initialization can fail.
Steps To Reproduce
nix-build the following derivation (ran out of disk space trying to build master, but looking at the source the issue seems to be the same, see additional info section). The issue happens both with and without emacs-overlay.
This will build a package my-emacs pulling in which-key and a package emacs-27.2 which is Emacs without the extra packages. The bin/emacs executable in my-emacs is a script setting up the environment so the wrapped executable will find which-key.
Run ./result/bin/emacs and press =C-h v invocation-directory= and check the resulting path. It should point inside the store path in target, but will be the path of the emacs-27.2 package.
Start a second Emacs instance from inside this one (variables need to be inserted by hand): M-x, shell-command, {invocation-directory}{invocation-name}
in the second Emacs instance try to load the which-key package: M-x, load-library, which-key
Expected behavior
The invocation-directory should point to the $my-emacs/bin/ instead of $emacs-27.2/bin in both Emacs instances.
Additional context
In nixpkgs I see wrapper.sh which does an exec without -a which seems to be where the original invocation name gets lost. This is the same on master. This will end up being the bin/emacs binary in the built derivation for my-emacs. bin/emacs in emacs-27.2 will be another wrapper doing another exec -a (I could not figure out where this was being generated, though).
I tried changing wrapper.sh using this patch but this was not successful, yet.
I think the second wrapper script in emacs-27.2 is overriding the invocation name, again.
Notify maintainers
@Stunkymonkey @alyssais @tadfisher (based on blame, not sure where the meta info is or if this is part of some specific package?)
Metadata
Please run
nix-shell -p nix-info --run "nix-info -m"and paste the result.Maintainer information: