Feature Description

As far as I have been able to discover, maintaining a complex "dependency tree" of Elpaca installations is convenient for use-package users, but not others.

It might make sense for Elpaca itself to provide a convenient way to manage non-trivial installation time dependencies between packages. I am running into situations where nested calls to the elpaca macro, whch appears to be Elpaca's only provided option (apart from the use-package integration) are inconvenient.

I made a guess as to the best solution to my problem in the Title above, but I don't mean to be prescriptive, with a single solution in mind. Not only that, but I would be happy with any solution that solves my problem reasonably, including being informed that I am doing everything wrong. :-)

Consider the following instructions found at https://github.com/golang/tools/blob/master/gopls/doc/emacs.md#loading-eglot-in-emacs for using go-mode with eglot:

;; Optional: load other packages before eglot to enable eglot integrations.
(require 'company)
(require 'yasnippet)

(require 'go-mode)
(require 'eglot)
(add-hook 'go-mode-hook 'eglot-ensure)

;; Optional: install eglot-format-buffer as a save hook.
;; The depth of -10 places this before eglot's willSave notification,
;; so that that notification reports the actual contents that will be saved.
(defun eglot-format-buffer-before-save ()
  (add-hook 'before-save-hook #'eglot-format-buffer -10 t))
(add-hook 'go-mode-hook #'eglot-format-buffer-before-save)

This "simple" serial (synchronous) code is easy to follow, and maintain.

When using use-package this is relatively easy to express in a way that retains the something close to the simplicity of the above approach, but with maximum install-time parallelism, using use-package's :after keyword. It would be something like this (untested, take these examples as pseudo code):

;; Install use-package support
(elpaca elpaca-use-package
  ;; Enable use-package :ensure support for Elpaca.
  (elpaca-use-package-mode))
(elpaca-wait) ;; FIXME: is this necessary?

(use-package company :ensure t)
(use-package yasnippet :ensure t)
(use-package go-mode
  :ensure t
  ;; hook up other Go mode initialization here)
(use-package eglot
  :ensure t
  :after (company yasnippet go-mode)
  :config
  (add-hook 'go-mode-hook 'eglot-ensure))

Without use-package, I believe one option is to use elpaca-wait to force an installation synchronization point:

(elpaca company)
(elpaca yasnippet)
(elpaca go-mode)
(elpaca-wait)  ; <==========
(elpaca eglot
  (add-hook 'go-mode-hook 'eglot-ensure))

This is mirrors the "serial" look of the first solution, but potentially misses out on parallelism if additional, unrelated, packages are installed in the config, after the (elpaca-wait).

Another alternative, suggested in the README at the IMPORTANT: section (seems to be an un-linkable heading):

(elpaca company
  (elpaca yasnippet
    (elpaca go-mode
      (elpaca eglot
        (add-hook 'go-mode-hook 'eglot-ensure)))))

setup.el

In truth, I'm not trying to express my config in vanilla elpaca and lisp, I'm using setup.el, which does have a deferred loading just as use-package does. See :load-after here: https://www.emacswiki.org/emacs/SetupEl#h5o-10

The problem is, Elpaca does not expose an equivalent to with-eval-after-load but for package installations.

The elpaca macro itself is not equivalent because:

It isn't clear if multiple calls to elpaca with the same ORDER is supported.
Even if they are, passing the same ORDER to multiple elpaca calls, if they are intended to install the same package, is unwieldy.

So it appears there is no way to express something like with-eval-after-elpaca-load PACKAGE_NAME semantics, which is what I think would be most convenient.

Confirmation

[X] The feature I'm proposing does not already exist in Elpaca

It might make sense for Elpaca itself to provide a convenient way to manage non-trivial installation time dependencies between packages.

Packages provide this information via the Package-Requires metadata in the main elisp file header.

situations where nested calls to the elpaca macro, whch appears to be Elpaca's only provided option (apart from the use-package integration) are inconvenient.

I do not recommend nesting elpaca declarations. That is potentially more confusing than it needs to be.

(elpaca-wait) ;; FIXME: is this necessary?

That should not be necessary on recent versions of Elpaca. By default, :wait t is inherited from the elpaca-use-package recipe:

https://github.com/progfolio/elpaca/blob/b8ed514119df6aa0e065dfdf8c4fa75f0b8802ca/elpaca.el#L174

Without use-package, I believe one option is to use elpaca-wait to force an installation synchronization point:
(elpaca company)
(elpaca yasnippet)
(elpaca go-mode)
(elpaca-wait)  ; <==========
(elpaca eglot
  (add-hook 'go-mode-hook 'eglot-ensure))

The BODY of each declaration in a queue is evaluated in declared order after the entire queue is finished installing/activating, so your example should work without the elpaca-wait call if all the configuration occurs in the BODY of each declaration. Note you'd still want to use with-eval-after-load if the configuration is load order dependent (that has nothing to do with the package manager of choice, though). e.g.

(elpaca eglot
  (with-eval-after-load "go-mode"
    (add-hook 'go-mode-hook #'eglot-ensure)))

The following test case should help illustrate. Notice the first declaration has a :pre-build step which introduces an artificial delay of 2 seconds. As shown in the log, its package installed last. However, the declaration's BODY was run before the second package's.

Test Case

[How to run this test?](https://github.com/progfolio/elpaca/wiki/Troubleshooting#the-elpaca-test-macro) ```emacs-lisp (elpaca-test :early-init (setq elpaca-menu-functions nil) :init (elpaca (wikinfo :host github :repo "progfolio/wikinfo" :pre-build ("sleep" "2")) (message "one")) (elpaca (doct :host github :repo "progfolio/doct") (message "two")) (elpaca-wait) (princ (elpaca-log "#unique"))) ```

Host Env

elpaca	b8ed514 HEAD -> master, origin/master, origin/HEAD
installer	0.7
emacs	GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.42, cairo version 1.18.0) of 2024-07-02
git	git version 2.45.2

Output

```emacs-lisp one two doct finished ✓ 0.548 secs elpaca finished ✓ 0.731 secs wikinfo finished ✓ 2.690 secs ```

Another alternative, suggested in the README at the IMPORTANT: section
(elpaca company
  (elpaca yasnippet
    (elpaca go-mode
      (elpaca eglot
        (add-hook 'go-mode-hook 'eglot-ensure)))))

There's no suggestion to nest elpaca declarations, only to move top-level code which needs to be run after a package is installed/activated into the BODY of a declaration or, alternatively, into a with-eval-after-load form.

Perhaps the setup macro needs to be altered or has a bug. I don't use setup.el or maintain its Elpaca integration, though, so you'd be better off asking someone more familiar with that particular macro.

It isn't clear if multiple calls to elpaca with the same ORDER is supported.

Doing so will result in a the warning described here: https://github.com/progfolio/elpaca/wiki/Warnings-and-Errors#duplicate-item-warnings

Does that help?

Thank you. It was not clear to me that the package guarantees that each elpaca BODY executes in the order established in by successive calls to the elpaca macro.

As I demonstrated by my confusion, I was not able to figure this out on my own. I have a few suggestions:

I would be happy to send a PR to improve documentation. First, though, I'd like to explain what confused me in case that helps me understand things better.

I think it would be good to improve the doc comment for the elpaca macro itself. Explain what "defer execution of BODY" means. Explain the order deferred bodies will execute relative to prior and subsequent calls to elpaca. Ideally, explain what an ORDER is.

The "IMPORTANT:" section in the README is not clear to me. I will explain why.

Elpaca installs and activates packages asynchronously. Elpaca processes its package queues after Emacs reads the init file.3

(elpaca package-a (message "First")) ; Queue First (message "Second") ; Second messaged (elpaca package-b (message "Third")) ; Queue Third (elpaca-process-queues) ; Process queue: First messaged, Third messaged.

..."Second" will be message before "First" and "Third"...

I was looking for an explicit claim that the body of (elpaca package-a ...) would always be evaluated before the body of (elpaca package-b ...), but I did not find it.

It also isn't clear what "processed" means. Is this to mean the BODY forms are executed? Some internal processing by Elpaca itself? These are questions I asked myself and came up with the wrong conclusions.

Defer forms which are dependent on deferred forms.

This sentence does not make sense to me. Is this a command I am supposed to follow? I don't understand it. I don't know what a "deferred form" is, and I don't know how I can "defer forms". I don't know when I would want to do this.

If the top-level form should be executed after a specific package is installed/activated, put it in that declaration's BODY. e.g.

This sentence is how I concluded that if I wanted one elpaca BODY to execute after another elpaca BODY, then I needed to nest them.

Again, I am happy to send a PR to improve the documentation, but I'd first like to be sure I understand things.

I think it would be good to improve the doc comment for the elpaca macro itself. Explain what "defer execution of BODY" means. Explain the order deferred bodies will execute relative to prior and subsequent calls to elpaca.

There's a balance I'd like to strike between dumping all this info in the docstrings and README and the manual. The manual is the place for details on overarching concepts as far as I'm concerned. Is this docstring any clearer for you?:

(defmacro elpaca (order &rest body)
  "Queue ORDER for asynchronous installation/activation.
Evaluate BODY forms synchronously once ORDER's queue is finished processing."
  (declare (indent 1) (debug form))
  `(elpaca--expand-declaration ',order ',body))

Ideally, explain what an ORDER is.

The manual has sections on orders, recipes, queues, menus, etc. Still a work in progress, but those are intended to explain the basic concepts:

https://github.com/progfolio/elpaca/blob/master/doc/manual.md#orders

I was looking for an explicit claim that the body of (elpaca package-a ...) would always be evaluated before the body of (elpaca package-b ...), but I did not find it.

It also isn't clear what "processed" means. Is this to mean the BODY forms are executed? Some internal processing by Elpaca itself? These are questions I asked myself and came up with the wrong conclusions.

Does the manual section on queues help at all?:

Elpaca installs packages asynchronously. Orders (*note orders: Orders.) are automatically queued in a list. When all of a queue's orders have either finished or failed Elpaca considers it "processed".

Queues ensure packages installation, activation, and configuration take place prior to packages in other queues. The ‘:wait’ recipe keyword splits the current queue and immediately begins processing prior queues. This is useful when one wants to use a package from a previous queue later on at the top-level of their init file. For example, a package which implements an Elpaca menu (*note menu: Menus.):
(elpaca (melpulls :host github :repo "progfolio/melpulls" :wait t)
  (add-to-list 'elpaca-menu-functions #'melpulls)
  (elpaca-update-menus #'melpulls)))

;; Implicitly queued into a new queue.
(elpaca menu-item-available-in-melpulls)
Defer forms which are dependent on deferred forms.

This sentence does not make sense to me. Is this a command I am supposed to follow? I don't understand it. I don't know what a "deferred form" is, and I don't know how I can "defer forms". I don't know when I would want to do this.

Honestly, it looks like a copy/paste error. Thanks for pointing it out. I've replaced it with a sentence clarifying how declaration BODY forms are evaluated synchronously (in declared order).

Do the manual and the changes made in 44b4b91 help?

I am embarrassed. I forgot that the manual existed!

In my defense, I have been suffering from a Covid induced fever the past few days, so I haven't been thinking clearly at times. Perhaps not the best time to be learning new things and making suggestions!

But also, I'm quite used to doc strings being both verbose and useful when learning new things in Emacs. For example, compare the doc string for auto-mode-alist and the discussion of same in the Emacs manual. Or, the doc string for find-file, which even describes basic use of Tramp. The manuals tend to be supersets of the doc strings, but they are typically not spartan.

The new docstring is an improvement.

I attempted to flesh it out with much more detail:

  "Queue ORDER and BODY for later processing.

The ORDER describes a package to be installed and actiavted.  Commonly,
ORDER can be a single symbol naming the package.  It can also be a full
or partial recipe, for more control over which package is installed and
how.  See the Info node `(elpaca) Menus' and the Info node `(elpaca)
Orders' in the Elpaca manual, for more about this.

The package install does not occur immediately.  Installs happen,
asynchonrosly, when one of:

 - The user's init file has been completely evaluated.
 - An ORDER with the `:wait' word is enqueued.
 - The `elpaca-wait' function is called.

When the installations and package activations are done then, then the
associated BODY forms are evaluated in order of the preceeding calls to
the `elpaca' macro.  See the Info node `(elpaca) Queues' in the Elpaca
manual, for more about this."

If I return to editing my Emacs config file after months of not thinking about Elpaca, I can do C-x f elpaca and see basically all I want to know to refresh my memory of how to use the macro. If I need more, the links to the manual will be one click away.

Does the manual section on queues help at all?:

Yes. Only one thing is not clear to me: the manual mentions "queues" (noun, plural) but I can't figure out how there can be more than one active queue.

Do the manual and the changes made in https://github.com/progfolio/elpaca/commit/44b4b91a67a65e165ceaa41cffb84f02b81ccd0f help?

Yes. The only nit I have is with this sentence:

If a top-level form should be executed after a specific package is installed/activated, put it in that declaration's BODY.

I was thrown by "top-level form". Can this simply be "If a form"? I would also say "evaluated" instead of "executed" to be more lispy (e.g. Emacs' eval, eval-buffer, with-eval-after-load, etc.).

I am embarrassed. I forgot that the manual existed!

Not a problem.

But also, I'm quite used to doc strings being both verbose and useful when learning new things in Emacs. For example, compare the doc string for auto-mode-alist and the discussion of same in the Emacs manual. Or, the doc string for find-file, which even describes basic use of Tramp. The manuals tend to be supersets of the doc strings, but they are typically not spartan.

The new docstring is an improvement.

I attempted to flesh it out with much more detail:

I prefer to keep most of the details in the manual. So I've added a link to that in the elpaca macro's docstring.

Yes. Only one thing is not clear to me: the manual mentions "queues" (noun, plural) but I can't figure out how there can be more than one active queue.

There is never more than one queue being processed at once.

Yes. The only nit I have is with this sentence:

Reworded to make it clearer.

progfolio / elpaca

[Feature]: Provide a first class `:after` without requiring use-package. #330

Feature Description

setup.el

Confirmation