AndersonTorres
commented
11 months ago My question is: why?
Not being hostile, but it is more than expected that Emacs distributions will not work with Nixpkgs Emacs framework.
Closed azahi closed 5 months ago
AndersonTorres
commented
11 months ago My question is: why?
Not being hostile, but it is more than expected that Emacs distributions will not work with Nixpkgs Emacs framework.
azahi
commented
11 months ago My question is: why?
Do you mean "why we should bother with configuration distributions"?
it is more than expected that Emacs distributions will not work with Nixpkgs Emacs framework
I didn't expect that. A lot of people use these frameworks (including me) and having support for them in Nixpkgs would be nice to have. To me this seems like a bug, why would emacs.pkgs.withPackages behavior differ for this kind of thing?
Also, this looks like a simple fix: I'm guessing load-path for emacs.pkgs.withPackages needs to be tweaked somehow, I'm not sure how to do it at this point, didn't have time to look through it. I just hoped that I can get some insight from the maintainers so I can potentially fix it myself.
AndersonTorres
commented
11 months ago Do you mean "why we should bother with configuration distributions"?
Yes. Are they Nixpkgs-friendly to begin with? They have a stable API to work with?
I didn't expect that.
Why not? Nixpkgs does a lot of heavy patching and dark magic in order to provide an useful experience for simple things as commandline compilers. Powerful text editors are way harder than this, with a lot of intermediate state (env vars, dotfiles, internal package managers...).
A lot of people use these frameworks (including me) and having support for them in Nixpkgs would be nice to have.
In practical terms, how much Nixpkgs should do in order to provide nice support for the plethora of starter kits? And who is interested in doing this?
azahi
commented
11 months ago Yes. Are they Nixpkgs-friendly to begin with? They have a stable API to work with?
What exactly makes software "Nixpkgs-friendly"? I think this line of thinking could be applied to other packages, not just Emacs configuration distributions. Isn't Nixpkgs itself already a bag of hacks for forever evolving software that it tries to package?
Forgive me if this is getting off-topic. You don't have to answer the above questions.
In practical terms, how much Nixpkgs should do in order to provide nice support for the plethora of starter kits? And who is interested in doing this?
I'm not implying that someone must dedicate their time to support all of them, I understand that's literally impossible to do, instead it would be simply nice to have support for issues that may arise (such as mine). At this point I only want to get some pointers on how to fix a specific problem. 😕
Feel free to close this issue if you think that this is not a correct place to ask for something like this. I can try my luck upstream or on our Discourse.
AndersonTorres
commented
11 months ago Let's be more practical.
We have a bug, namely, "doom and nixpkgs are not working together".
Supposing this is because Nixpkgs is the buggy part, the typical course of action is to reproduce this bug with a minimal configuration - extra points for one without doom.
If the buggy part is doom, we can't do much except maybe send a log file to the doom maintainers.
(Iirc hlissner is a nixpkgs enthusiast, maybe they is interested.)
nagy
commented
11 months ago For me, emacs.pkgs.withPackages works fine with doom. But I have to remind myself to run doom sync whenever I have new packages given to withPackages. Maybe try to doom install with an empty emacs, as you already did, and then doom sync afterwards with your package filled emacs.
azahi
commented
11 months ago Supposing this is because Nixpkgs is the buggy part, the typical course of action is to reproduce this bug with a minimal configuration - extra points for one without doom.
Makes sense, I'll try to reproduce it with a minimal config later.
Maybe try to doom install with an empty emacs, as you already did, and then doom sync afterwards with your package filled emacs.
I did try running nix shell nixpkgs#emacs and installed Doom successfully with it, but subsequent invocations of the doom script yields the same error if the emacs binary is used from withPackages derivation. Also I've got a feeling that packages installed in withPackages are not picked up because I had to manually recompile libvterm, which supposed to be already compiled for emacs.pkgs.vterm. But I'm guessing that's outside of the scope for this issue.
Here's what I dug out this far:
[azahi@eonwe:~]$ cat /nix/store/22z7ja19nhvwp8lh3j5q4xrc3jr1ckjj-emacs-packages-deps/share/emacs/site-lisp/site-start.el
(let ((inhibit-message t))
(load "/nix/store/4sbqs4ksaf7kcv0nh96s0qwkig4zxzcq-emacs-with-packages-29.1/share/emacs/site-lisp/site-start"))
;; "/nix/store/22z7ja19nhvwp8lh3j5q4xrc3jr1ckjj-emacs-packages-deps/share/emacs/site-lisp" is added to load-path in wrapper.sh
;; "/nix/store/22z7ja19nhvwp8lh3j5q4xrc3jr1ckjj-emacs-packages-deps/share/emacs/native-lisp" is added to native-comp-eln-load-path in wrapper.sh
(add-to-list 'exec-path "/nix/store/22z7ja19nhvwp8lh3j5q4xrc3jr1ckjj-emacs-packages-deps/bin")
[azahi@eonwe:~]$ file /nix/store/4sbqs4ksaf7kcv0nh96s0qwkig4zxzcq-emacs-with-packages-29.1/share/emacs/site-lisp/site-start
/nix/store/4sbqs4ksaf7kcv0nh96s0qwkig4zxzcq-emacs-with-packages-29.1/share/emacs/site-lisp/site-start: cannot open `/nix/store/4sbqs4ksaf7kcv0nh96s0qwkig4zxzcq-emacs-with-packages-29.1/share/emacs/site-lisp/site-start' (No such file or directory)
[azahi@eonwe:~]$ ls -lha /nix/store/22z7ja19nhvwp8lh3j5q4xrc3jr1ckjj-emacs-packages-deps/bin
total 1.0K
dr-xr-xr-x 2 root root 2 Jan 1 1970 .
dr-xr-xr-x 4 root root 4 Jan 1 1970 ..Looks like this part generates a load that points to a non-existent file when there's withPackages involved. Plain emacs has a proper site-lisp, which I'm guessing already comes bundled with Emacs. Will this be enough to warrant a fix for Nixpkgs?
As a side note, I recon this is the offending part for Doom. Deleting this part makes the doom script work. It also looks like only the doom CLI utility is affected because when I simply run emacs the error does not occur.
azahi
commented
11 months ago Judging by the code, Doom makes the assumption that the first load-path directory has the site-lisp inside it but withPackages doesn't provide it. Do you think that complying with that and adding/linking XXXX-emacs-packages-deps/share/emacs/site-lisp to XXXX-emacs-with-packages-29.1/share/emacs/site-lisp derivation would be a good idea?
AndersonTorres
commented
11 months ago It resumes to ask if withPackages should provide site-lisp.
On the other hand, I don't know if other packages would break if we provide site-lisp this way.
IMHO it looks reasonable to do what you suggested.
farra
commented
10 months ago I just hit the exact same issue. Below is simple confirmation and some steps for reproducing.
Context Relatively new to nix and home-manager. Attempting to setup a new machine / environment using home-manager. Intention was to setup emacs29 + doom emacs. Approach was to install emacs29 via nix/home-manager, then install doom emacs manually. Rational is based on the sense that nix-doom-emacs was not necessarily stable and based on this 2 year old reddit thread.
Reproducing
Using a new home-manager installation, configure home.nix like this:
https://www.reddit.com/r/emacs/comments/15opqdy/installing_emacs_on_macos_with_nix_or_homebrew/
Specifically, when using the following in home.nix
let
emacs-overlay = import (fetchTarball {
url = "https://github.com/nix-community/emacs-overlay/archive/master.tar.gz";
sha256 = "1jppksrfvbk5ypiqdz4cddxdl8z6zyzdb2srq8fcffr327ld5jj2";
});
my-emacs = pkgs.emacs29.override {
withNativeCompilation = true;
withSQLite3 = true;
withTreeSitter = true;
withWebP = true;
};
my-emacs-with-packages = (pkgs.emacsPackagesFor my-emacs).emacsWithPackages (epkgs: with epkgs; [
pkgs.mu
vterm
multi-vterm
pdf-tools
treesit-grammars.with-all-grammars
]);
in
{
programs.emacs = {
enable = true;
package = my-emacs-with-packages;
};Then attempt to run:
git clone --depth 1 https://github.com/doomemacs/doomemacs ~/.config/emacs
~/.config/emacs/bin/doom installYou will get the error that @azahi posted.
However, if I changed the above block to:
let
emacs-overlay = import (fetchTarball {
url = "https://github.com/nix-community/emacs-overlay/archive/master.tar.gz";
sha256 = "1jppksrfvbk5ypiqdz4cddxdl8z6zyzdb2srq8fcffr327ld5jj2";
});
my-emacs = pkgs.emacs29.override {
withNativeCompilation = true;
withSQLite3 = true;
withTreeSitter = true;
withWebP = true;
};
my-emacs-with-packages = (pkgs.emacsPackagesFor my-emacs).emacsWithPackages (epkgs: with epkgs; [
pkgs.mu
vterm
multi-vterm
pdf-tools
treesit-grammars.with-all-grammars
]);
in
{
programs.emacs = {
enable = true;
package = my-emacs;
};Note I'm now using package = my-emacs, not package = my-emacs-with-packages
With that change, then doom emacs installs correctly.
Comment There aren't many great resources online for this yet. AFAIK, there hasn't yet emerged a set of best practices for using home-manager and doom emacs. This is my second attempt at using home-manager. The first was 2 years ago and I abandoned it shortly thereafter. It appears home-manager is more polished and stable now, but these sort of errors would certainly discourage adoption of what are fairly common configurations.
jian-lin
commented
10 months ago (load "/nix/store/4sbqs4ksaf7kcv0nh96s0qwkig4zxzcq-emacs-with-packages-29.1/share/emacs/site-lisp/site-start"))
@azahi Is it a typo? It should be (load "nix/store/hash-emacs-version/share/emacs/site-lisp/site-start") and it is on my machine.
jian-lin
commented
10 months ago I am interested in this issue. However, I am not familiar with Doom. Is there a way to run doom install in a test environment without changing my daily emacs config?
nagy
commented
10 months ago Is there a way to run doom install in a test environment without changing my daily emacs config?
$ cd $(mktemp -d)
$ export HOME=$PWDAlso just fyi, this https://github.com/nix-community/nix-doom-emacs is a project that attempts to solve the doom installation declaratively.
azahi
commented
10 months ago @azahi Is it a typo?
No, it's not. This is how I got this derivation:
pkgs.emacs29.pkgs.withPackages (p: with p; [emacsql mu4e vterm])TL;DR: do not double wrap your Emacs.
The Emacs infrastructure in Nixpkgs assumes that you only wrap your Emacs once. Double wrapping may cause issues:
In fact, it is a coincidence that double wrapping works in certain circumstances.
The reason for this issue:
subdirs.el again for some reason I am unawaresubdirs.el generated by the wrapper is not semantically equal to the original one. Our optimized one is not idempotent while the original one is. When you double wrap your Emacs, the second run of our optimized subdirs.el, caused by doom as I mention above, changes the order of load-path, which causes this issue.AndersonTorres
commented
10 months ago Indeed, double wrapping is a serious issue in general around Nixpkgs.
collares
commented
8 months ago For future reference, if you're using home-manager, extra packages should be added like this to avoid double wrapping:
programs.emacs = {
enable = true;
package = YOUR-EMACS-PACKAGE; # e.g. pkgs.emacs29
extraPackages = (epkgs: [ epkgs.PACKAGE1 epkgs.PACKAGE2 ]);
};
DamienCassou
commented
5 months ago I don't think there is anything to solve anymore so I'm closing the issue. Feel free to reopen.
Describe the bug
Doom Emacs does not work with
emacs.pkgs.withPackages.Steps To Reproduce
Steps to reproduce the behavior:
Error: doom-core-error ("/home/azahi/.config/emacs/lisp/doom-cli.el" (file-missing "Cannot open load file" "No such file or directory" "/nix/store/z4khv4j54as6wksi55cazciz8pdv169h-emacs-with-packages-29.1/share/emacs/site-lisp/site-start")) mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode 0x1fdcd16b59aef0e7>)) debug-early-backtrace() debug-early(error (doom-core-error "/home/azahi/.config/emacs/lisp/doom-cli.el" (file-missing "Cannot open load file" "No such file or directory" "/nix/store/z4khv4j54as6wksi55cazciz8pdv169h-emacs-with-packages-29.1/share/emacs/site-lisp/site-start"))) signal(doom-core-error ("/home/azahi/.config/emacs/lisp/doom-cli.el" (file-missing "Cannot open load file" "No such file or directory" "/nix/store/z4khv4j54as6wksi55cazciz8pdv169h-emacs-with-packages-29.1/share/emacs/site-lisp/site-start"))) (condition-case e (load path noerror 'nomessage) ((debug doom-error) (signal (car e) (cdr e))) ((debug error) (setq path (locate-file path load-path (get-load-suffixes))) (signal (cond ((not (and path (featurep 'doom))) 'error) ((file-in-directory-p path (expand-file-name "cli" doom-core-dir)) 'doom-cli-error) ((file-in-directory-p path doom-core-dir) 'doom-core-error) ((file-in-directory-p path doom-user-dir) 'doom-user-error) ((file-in-directory-p path doom-profile-dir) 'doom-profile-error) ((file-in-directory-p path doom-modules-dir) 'doom-module-error) ('doom-error)) (list path e)))) doom-load("doom-cli" nil) (or (featurep feature subfeature) (doom-load (if subfeature (file-name-concat doom-core-dir (string-remove-prefix "doom-" (symbol-name feature)) (symbol-name filename)) (symbol-name feature)) noerror)) (let ((subfeature (if (symbolp filename) filename))) (or (featurep feature subfeature) (doom-load (if subfeature (file-name-concat doom-core-dir (string-remove-prefix "doom-" (symbol-name feature)) (symbol-name filename)) (symbol-name feature)) noerror))) doom-require(doom-cli) (if (let ((load-suffixes '(".elc" ".el"))) (condition-case (load (expand-file-name "lisp/doom" user-emacs-directory) nil (not init-file-debug) nil 'must-suffix) (file-missing (prog1 (defalias 'startup--load-user-init-file@reroute-to-profile #'(lambda (args) (list #'(lambda nil (expand-file-name "init.el" user-emacs-directory)) nil (nth 2 args)))) (advice-add 'startup--load-user-init-file :filter-args #'startup--load-user-init-file@reroute-to-profile)) (setq user-init-file (expand-file-name "early-init" user-emacs-directory)) (setq load-prefer-newer t) (setq gc-cons-threshold ( 16 1024 1024)) nil))) (doom-require (if noninteractive 'doom-cli 'doom-start))) (let (file-name-handler-alist) (let ((command-line-args (if noninteractive nil command-line-args)) (profile (or (car (cdr (member "--profile" command-line-args))) (getenv-internal "DOOMPROFILE")))) (if (null profile) (let ((init-dir (or (car (cdr (member "--init-directory" command-line-args))) (getenv-internal "EMACSDIR")))) (if (null init-dir) (if noninteractive (progn (setq user-emacs-directory (file-name-directory (file-truename load-file-name))))) (setq command-switch-alist (cons (cons "--init-directory" #'(lambda () (car-safe (prog1 argv (setq argv (cdr argv)))))) command-switch-alist)) (setq user-emacs-directory (expand-file-name init-dir)))) (setq command-switch-alist (cons (cons "--profile" #'(lambda () (car-safe (prog1 argv (setq argv (cdr argv)))))) command-switch-alist)) (setenv "DOOMPROFILE" profile) (or (load (expand-file-name (format (let ((lfile (getenv-internal "DOOMPROFILELOADFILE"))) (if lfile (concat (string-remove-suffix ".el" lfile) ".%d.elc") "profiles/load.%d.elc")) emacs-major-version) user-emacs-directory) 'noerror (not init-file-debug) 'nosuffix) (user-error "Profiles not initialized yet; run 'doom sync' first")))) (if (let ((load-suffixes '(".elc" ".el"))) (condition-case (load (expand-file-name "lisp/doom" user-emacs-directory) nil (not init-file-debug) nil 'must-suffix) (file-missing (prog1 (defalias 'startup--load-user-init-file@reroute-to-profile #'(lambda (args) (list #'(lambda nil (expand-file-name "init.el" user-emacs-directory)) nil (nth 2 args)))) (advice-add 'startup--load-user-init-file :filter-args #'startup--load-user-init-file@reroute-to-profile)) (setq user-init-file (expand-file-name "early-init" user-emacs-directory)) (setq load-prefer-newer t) (setq gc-cons-threshold ( 16 1024 1024)) nil))) (doom-require (if noninteractive 'doom-cli 'doom-start)))) (or (let (file-name-handler-alist) (let ((command-line-args (if noninteractive nil command-line-args)) (profile (or (car (cdr (member "--profile" command-line-args))) (getenv-internal "DOOMPROFILE")))) (if (null profile) (let ((init-dir (or (car (cdr (member "--init-directory" command-line-args))) (getenv-internal "EMACSDIR")))) (if (null init-dir) (if noninteractive (progn (setq user-emacs-directory (file-name-directory (file-truename load-file-name))))) (setq command-switch-alist (cons (cons "--init-directory" #'(lambda () (car-safe (prog1 argv (setq argv (cdr argv)))))) command-switch-alist)) (setq user-emacs-directory (expand-file-name init-dir)))) (setq command-switch-alist (cons (cons "--profile" #'(lambda () (car-safe (prog1 argv (setq argv (cdr argv)))))) command-switch-alist)) (setenv "DOOMPROFILE" profile) (or (load (expand-file-name (format (let ((lfile (getenv-internal "DOOMPROFILELOADFILE"))) (if lfile (concat (string-remove-suffix ".el" lfile) ".%d.elc") "profiles/load.%d.elc")) emacs-major-version) user-emacs-directory) 'noerror (not init-file-debug) 'nosuffix) (user-error "Profiles not initialized yet; run 'doom sync' first")))) (if (let ((load-suffixes '(".elc" ".el"))) (condition-case _ (load (expand-file-name "lisp/doom" user-emacs-directory) nil (not init-file-debug) nil 'must-suffix) (file-missing (prog1 (defalias 'startup--load-user-init-file@reroute-to-profile #'(lambda (args) (list #'(lambda nil (expand-file-name "init.el" user-emacs-directory)) nil (nth 2 args)))) (advice-add 'startup--load-user-init-file :filter-args #'startup--load-user-init-file@reroute-to-profile)) (setq user-init-file (expand-file-name "early-init" user-emacs-directory)) (setq load-prefer-newer t) (setq gc-cons-threshold ( 16 1024 1024)) nil))) (doom-require (if noninteractive 'doom-cli 'doom-start)))) (load user-init-file 'noerror (not init-file-debug) nil 'must-suffix)) load-with-code-conversion("/home/azahi/.config/emacs/early-init.el" "/home/azahi/.config/emacs/early-init.el" nil t) load("/home/azahi/.config/emacs/early-init.el" nil nomessage nosuffix) (and (load init-file nil 'nomessage 'nosuffix) (featurep 'doom)) (or (and (load init-file nil 'nomessage 'nosuffix) (featurep 'doom)) (user-error "Failed to load Doom from %s" init-file)) (let ((bin-dir (file-name-directory (file-truename load-file-name))) (init-file (expand-file-name "../early-init.el" bin-dir))) (or (and (load init-file nil 'nomessage 'nosuffix) (featurep 'doom)) (user-error "Failed to load Doom from %s" init-file))) (condition-case e (let* ((bin-dir (file-name-directory (file-truename load-file-name))) (init-file (expand-file-name "../early-init.el" bin-dir))) (or (and (load init-file nil 'nomessage 'nosuffix) (featurep 'doom)) (user-error "Failed to load Doom from %s" init-file))) (user-error (message "Error: %s" (car (cdr e))) (kill-emacs 2))) load-with-code-conversion("/home/azahi/.config/emacs/bin/doom" "/home/azahi/.config/emacs/bin/doom" nil t) command-line-1(("--load" "/home/azahi/.config/emacs/bin/doom" "--" "install")) command-line() normal-top-level() Unexpected error in Doom’s core: "/home/azahi/.config/emacs/lisp/doom-cli.el", (file-missing "Cannot open load file" "No such file or directory" "/nix/store/z4khv4j54as6wksi55cazciz8pdv169h-emacs-with-packages-29.1/share/emacs/site-lisp/site-start")
Screenshots
Nope.
Additional context
Nah.
Notify maintainers
@AndersonTorres @adisbladis @Atemu @jwiegley @lovek323 @matthewbauer
Metadata
"x86_64-linux"Linux 6.1.61-xanmod1, NixOS, 23.11 (Tapir), 23.11.20231107.ec750fdyesyesnix-env (Nix) 2.17.1/nix/store/mj0hy52z22q5gpsf33akndxiclxd8ray-source