posts/2021-04-09-emacs-d

[utterances-bot]

Towards future-safe emacs.d

Boosting emacs.d development flow

https://d12frosted.io/posts/2021-04-09-emacs-d.html

[joyguar]

Hello, I have found your guide very helpful and an inspiration for my own modular configuration of Emacs based on similar principles. After creating the init, Eldev, and init-elpa files, I also created several packages in the lisp directory. The eldev build :autoloads step works fine, but eldev compile returns an error for those custom packages that require packages downloaded through straight.el. More specifically, I get the error "Cannot load xxx: (file-missing "Cannot open load file" "No such file or directory" "xxx"). I believe this is because the straight directory is not in the load-path. I notice that no where in your guide, nor in the files discussed do you update the load-path to include these packages. So how do you avoid this error?

[d12frosted]

Hey @danielclucas

I am glad that you find it helpful. Regarding your question, section about content of init-elpa explains how to configure straight.el, which is responsible for load-path stuff. Just make sure that init.elpa is required from init.el file.

In case everything is all set and it still doesn't work, I would appreciate a link to repo/branch where I can see everything and try it myself.

[joyguar]

Thank you for your response, I am only just now getting back to this. I've updated my init-elpa file so that things should (hopefully) build correctly, but I am now running into an issue with the building of straight.el related to https://github.com/radian-software/straight.el/issues/946. I changed the bootstrap code to reflext the change from "raxod502" to "radian-software," but now attempts to build straight.el are thwarted by the fact that straight.el's remote origin has a URL pointing to radian-software but the recipe specifies the raxod502 URL. Do you know of a way to address this?

[d12frosted]

@danielclucas I haven't run into any issues related to https://github.com/radian-software/straight.el/issues/946. My guess would be to start Emacs directly, wait for this popup to appear, hit d and then either wait for everything to continue, or kill Emacs and run whatever command you were running from command line again 😄

[joyguar]

Ah, I hadn't realized that I could run that command from inside emacs even without the build having completed successfully. That did the trick! Everything has also compiled successfully. Now I will see if everything continues to work as I add all the nuts and bolts of my configuration!

[qingshuizheng]

Recently (maybe from the beginning of the 'eru install emacs' journey), I run into some frustrations when init-autoloads.el fails to build successfully, but the config runs okay if previously working version is not cleaned by 'eldev clean autoloads'.

Is there an option to backup the previous init-autoloads.el to elsewhere before 'eldev clean', and automatically rollback if the new autoload file fails to build?

Rollback is not a must-have one, but backup is. So at least there's backup to use.

Thanks! Zheng

[d12frosted]

Is there an option to backup the previous init-autoloads.el to elsewhere before 'eldev clean', and automatically rollback if the new autoload file fails to build?

@qingshuizheng There is no such option, but you can do it yourself. It depends on how you run these commands, you can always do a backup/restore flow there. But there is reason why this option is not available. Having an outdated autoloads file might lead to missing definitions in the 'runtime'. So IMO the most important question - why does it fail? Can you share some details?

[qingshuizheng]

So IMO the most important question - why does it fail?

I revamp my config quite often, and regularly build emacs and pull packages. I believe any of these actions may push the emacs ecosystem to the edge that it might fail with or without my awareness. When the autoloads failed to build successfully, emacs configs just won't work at all.

Having an outdated autoloads file might lead to missing definitions in the 'runtime'.

That makes sense. From my immature perspective, when autoloads.el fails to build, then I just want to get emacs work like usual temporarily :-), even with outdated definitions. Debug with no personal configs is a tortue.

you can always do a backup/restore flow there.

I think I can figure out something to copy the autoloads to elsewhere as a backup for rescue.

Thanks!

[d12frosted]

I revamp my config quite often, and regularly build emacs and pull packages. I believe any of these actions may push the emacs ecosystem to the edge that it might fail with or without my awareness. When the autoloads failed to build successfully, emacs configs just won't work at all.

I was rather curios about the error. In my understanding, autoloads generation should not fail unless you intentionally break it.

BTW, if you are using something like the following, you can simply remove these lines and you will be able to start without autoloads:

https://github.com/d12frosted/environment/blob/a85cd9c27615930162a3465c2ddba9b56b4f8316/emacs/init.el#L70-L74

[qingshuizheng]

In my understanding, autoloads generation should not fail unless you intentionally break it.

Any common :error level would break it, which makes sense.

Some upstreams issues are quite annoying, I found no solution until they are fixed.

The most recent one was, undo-fu.el and undo-fu-session.el moved repo to https://codeberg.org, and straight.el had not supported it yet, so autoload failed to build. (Now it got fixed: https://github.com/radian-software/straight.el/pull/992/commits/bddf4c7)

Another one is, straight doesn't build org correctly for some reason. And it drove me crazy. Sorry I forgot the details and am not able to provide any.

BTW, if you are using something like the following, you can simply remove these lines and you will be able to start without autoloads

Actually I move this section to the end of init.el, so if autoloads fails to build, at least most of the packages are usable and emacs are partially functioning.

[d12frosted]

Wait, init-autoloads.el that I describe here has nothing to do with packages. It contains autoloads of custom configurations and not packages, as the latter is handled by straight.el. So I am confused now.

Actually I move this section to the end of init.el, so if autoloads fails to build, at least most of the packages are usable and emacs are partially functioning.

But why do you need it then? I put it there to prevent myself from starting Emacs without autoloads. If you say that you are fine with running Emacs without your own autoloads, why do you need to error out?

[qingshuizheng]

Wait, init-autoloads.el that I describe here has nothing to do with packages. It contains autoloads of custom configurations and not packages, as the latter is handled by straight.el.

Take lib-vulpea.el for example, it requires svg-tag-mode to work, if straight.el fails to pull/build it for any reason, then the whole process of building init-autoloads.el would come to a stop. In this aspect, it relates to packages, but that's not your approach's fault, lol... That's why I'm here to ask if there's any implementation for backup and rollback.

(unless elpa-bootstrap-p
  ;; (unless (file-exists-p path-autoloads-file)
  ;;   (error "Autoloads file doesn't exist, please run '%s'"
  ;;          "eru install emacs"))
  (load path-autoloads-file t 'nomessage))

I put it at the end of file, so I have some basic packages to use, wether init-autoloads.el builds or not. It might have some side effect I don't notice, but seems haven't cause any major issues.

Sorry for the wrong usage of your approach to emacs-config. I'm totally a counter example of user. lol..

Will try to refork and revamp the config again.

[d12frosted]

🤔 This is exactly why the process of package installation is a separate step before autoloads in my flow. First I run make bootstrap, which fetches and installs all packages. And only then I run make compile which generates init-autoload.el file (among other operations).

Sorry for the wrong usage of your approach to emacs-config. I'm totally a counter example of user. lol..

Don't worry about that. I shared my approach, someone is using it, have questions - that's great. We have this communication medium for a reason, right? :) And btw I am not sure that you are counter example, I am just trying to better understand your flow.

[joyguar]

@d12frosted I have run into a very strange issue, I hope that perhaps you might have insight or a suggestion of what might be causing it. I recently did a fresh install of Arch, but after installing Emacs, I have run into issues with calling commands from my user-installed packages. Here is a typical example that pops up when I run eldev build :autoloads:

Error (use-package): doom-themes/:config: Cannot open load file: No such file or directory, ../../../../../../../../../../.config/flux-emacs/.local/packages/29.0/straight/build/doom-themes/doom-themes-ext-visual-bell`

It seems that when use-package is calling doom-themes-ext-visual-bell the function call is looking up that function with a very strange directory name. When I call a function from one of my installed packages, e.g. (execute-extended-command org-roam-node-find) I get a similar error where execute-extended-command is looking in ./../../../../../../../../../.config/flux-emacs/.local/packages/29.0/straight/build/... for the package and function.

By comparison Doom Emacs doesn't seem to have any of the same issues. I've tested this out with Emacs 29 + gtk + native comp, Emacs 29, Emacs 28, but they all have the same issue. I am not sure why Emacs is looking in this strange location when I invoke execute-extended-command. I attempted to step through execute-extended-command with edebug-defun and find out how the source of the command is resolved, but this seems to be handled still be execute-command, which I can't use edebug-defun on to step through as I have to use execute-command to execute some other command like org-roam-node-find which then triggers a backtrace but prevents me from stepping through... :sweat_smile:

I am currently at a loss, any help would be most appreciated! Here is my config for your reference https://github.com/danielclucas/flux-emacs

[d12frosted]

@danielclucas not sure if I have a remedy, but let's delve into it.

It seems that when use-package is calling doom-themes-ext-visual-bell the function call is looking up that function with a very strange directory name.

use-package doesn't call any doom-theme functions. You just instructed it to load doom-theme with the following stuff to be evaluated on load:

 (doom-themes-visual-bell-config)
 (doom-themes-org-config)

I am not familiar with doom-themes, but after skimming through their sources, it seems that doom-themes-visual-bell-config is defined in doom-themes-ext-visual-bell.el, and it's marked as autoloaded function. So what happens: you instruct to load doom-themes → your custom :config code is evaluated → call doom-themes-visual-bell-config → autoloading mechanism looks for implementation and since it's defined in doom-themes-ext-visual-bell.el it tries to load it. But it fails to find it.

Though why can't it find? I find it really strange that it complains specifically about 'extension' file and not about doom-theme.el. Are you sure you fully build packages? See my previous comment where I say that "package installation is a separate step before autoloads". You need to build packages before building your custom autoloads.

When I install doom-themes package, my build/doom-themes directory contains (among other things) the following files:

lrwxr-xr-x 1 d12frosted staff  106 Nov 25 08:33 doom-themes-ext-visual-bell.el -> /Users/d12frosted/.cache/emacs/packages/29.0/straight/repos/themes/extensions/doom-themes-ext-visual-bell.el
-rw-r--r-- 1 d12frosted staff  968 Nov 25 08:33 doom-themes-ext-visual-bell.elc

What do you have in that directory? Is it empty?

[joyguar]

This is what I have in that directory (among other things)

lrwxrwxrwx 1 trismegistus trismegistus   122 Nov 24 19:34 doom-themes-ext-visual-bell.el -> /home/trismegistus/.config/flux-emacs/.local/packages/29.0/straight/repos/themes/extensions/doom-themes-ext-visual-bell.el
-rw-r--r-- 1 trismegistus trismegistus   968 Nov 24 19:34 doom-themes-ext-visual-bell.elc

To your earlier point, when I perform eldev build :autoloads for the first time in my flux-emacs directory it builds the packages without issue. The issue appears to be that the autoloading mechanism is looking in a strange directory, ../../../../../../../../../../.config/flux-emacs/.local/packages/29.0/straight/build/doom-themes/doom-themes-ext-visual-bell Note that I'm not abbreviating the parent directories with /../, that is verbatim where it is looking, when I think it should be looking in /home/trismegistus/.config/flux-emacs/.local/packages/29.0/straight/build/doom-themes

[d12frosted]

@danielclucas Just tried to use your repository:

$ cd ~/.config
$ git clone git@github.com:danielclucas/flux-emacs.git
$ cd flux-emacs
$ make bootstrap
... wait for it to finish
$ make bootstrap
Wrote /Users/d12frosted/.config/flux-emacs/versions/default.el
[00:05.896]  Contents of package archive ‘melpa-unstable’ has been fetched already
[00:05.975]  Not fetching contents of other archive(s) as redundant
[00:05.975]  All project dependencies (including those for set ‘build’) have been installed already or are local
[00:05.975]  Generating build target list for standard fileset ‘main’
[00:05.989]  Reading target dependencies from file ‘.eldev/29.0/target-dependencies.build’...
[00:05.989]  Building plan: ‘lisp/init-autoloads.el’, ‘:autoloads’
[00:05.989]  AUTOLOADS -> lisp/init-autoloads.el
[00:06.007]  Loading file ‘lisp/init-autoloads.el’
[00:06.007]  Done building “sources” for virtual target ‘:autoloads’
[00:06.007]  Saving target dependencies to file ‘.eldev/29.0/target-dependencies.build’...
[00:06.008]  Finished successfully on Fri Nov 25 09:49:25 2022

Error in kill-emacs-hook (org-clock-save): (file-missing "Opening output file" "No such file or directory" "/Users/d12frosted/organum/org-clock-save.el")
Cleaning up the recentf list...
Cleaning up the recentf list...done (0 removed)

So it works for me. Maybe try cleaning build directory?

[joyguar]

I have deleted my .local file and tried building again, but the issue persists. I've also reinstalled eldev without any change.

This isn't just a use-package problem, as I referenced above, if I attempt to use a command provided by an installed package from within Emacs after building it such as org-roam-node-find I get

execute-extended-command: Cannot open load file: No such file or directory, ../../../../../../../../../../.config/flux-emacs/.local/packages/29.0/straight/build/org-roam/org-roam-node

I attempted to step through execute-extended-command but the error arises when command-execute is called from within execute-extended-command.

It seems to me that somehow, the way commands are being looked up from within command-execute is with this strange directory prefix.

[joyguar]

An update, I did a fresh install and the problem has resolved itself, still no idea what went wrong, but oh well! Thanks for the assistance!

[d12frosted]

@danielclucas not sure why it happens. Would be interesting to see your load path and content of autoloads. Oh, and check your custom.el file as it might be responsible for some strange stuff.

An update, I did a fresh install and the problem has resolved itself, still no idea what went wrong, but oh well! Thanks for the assistance!

Great to hear 😸