Integration with use-package, setup.el, leaf, etc

[arkhan]

Hello,

What a great job you are doing with this package manager, I have a query: How elpaca can be integrated with setup.el as an alternative to use-package.

Thank you very much

[progfolio]

The current state of affairs is that a wrapper-macro, elpaca-use-package is used to defer calls to use-package. This is inflexible, because users (as @reuleaux pointed out in #8 and @arkhan points out above) will want to use alternatives to use-package such as leaf or setup.el.

Code registered in an Elpaca queue is executed during after-init-hook. This is why the following does not work:

;;bootstrapping snippet omitted 
(elpaca my-preferred-macro (require 'my-preferred-macro)) ;; Queued, but has not been processed yet.
(my-preferred-macro etc) ;; This will fail because it is not in Elpaca's queue system and my-preferred-macro has not been installed/activated yet.

Or a simpler example:

;;bootstrapping snippet omitted 
(elpaca nil (message "second"))
(message "first")

Will message "first" while the init file is being read and "second" after the Elpaca queue has been processed.

Direct support for use-package is in the works. When it is finished it will make use of a general idiom that can be applied to leaf, setup.el, etc.

I will keep update this issue once I have a working solution and will provide specific examples. Thanks again for your patience and feedback, everyone.

[reuleaux]

OK, thanks so far.

Just one more bit: my very basic (straight) leaf config (for leaf itself) looks like this:

(eval-and-compile
  (straight-use-package 'leaf)
  (straight-use-package 'leaf-keywords)
  )

(leaf leaf
  :init
  (leaf leaf-keywords
    :init
    (leaf-keywords-init)
    )
  )

(eval-when-compile (msg "leaf done."))

Afterwards I can start using leaf like this:

(eval-and-compile
  (straight-use-package 'vertico)
  )

(leaf vertico
  :init
  (vertico-mode)
  ;; ... and more stuff, of course 
 )

(eval-when-compile (msg "vertico done."))

I am mentioning this just for (upcoming) testing purposes - just because it can be a little confusing, how leaf and leaf-keywords interact. - I got this basic nested leaf config from the leaf author at the time - and it still works for me (I assume you could have figured that out by looking at leaf and friends yourself).

Anyway, something like this in elpaca would be very much appreciated.

And thanks for everything so far: for what I can see (when I see elpaca running), this project looks very promising. - Keep up your good work. -A

[reuleaux]

...and for some nice leaf configuration examples have a look at https://www.grugrut.net/posts/my-emacs-init-el/ maybe - these may be easier to read than the (more reference style) leaf docs - thanks again

[Riyyi]

I have added a section to the setup.el wiki on how to use Elpaca with it. It relies on processing the initial queue with setup in it. This is currently done by just calling (elpaca-process-queues) very early in your init.el, which is kind of how the generic idiom will work in the future, as far as I can tell (this is from the feat/use-package-support branch): (elpaca-wait) ;process the previous queue so that top-level use-package declarations work

The wiki entry: Setup El#Using Elpaca

[progfolio]

@Riyyi:

Thank you for spreading the word.

Unfortunately, that configuration will not work. Setup will be correctly queued and will begin installing once elpaca-process-queues is called. However, the installation will be non-blocking, so by the time the interpreter reaches the first call to the setup macro it's not guaranteed to be installed and will most likely fail.

I haven't merged the use-package-support branch just yet because I still have to figure out the idiom for these use cases. I experimented with a top-level function, elpaca-wait, which would simply block until the current queues finished processing. Several different implementations turned out unsatisfactory, so I don't think I'll go that route. The next experiment will be a macro which properly nests queues.

(elpaca-with-queues
  (elpaca use-package (elpaca-use-package-mode))
  :then
  (use-package example :elpaca t))

Still haven't worked this out, but I'll take another swing at it soon.

[arkhan]

@Riyyi I try this:

;;; init.el --- -*- lexical-binding: t; -*-

;; -----------------------------------------
;; Bootstrap Elpaca
(declare-function elpaca-generate-autoloads "elpaca")
(defvar elpaca-directory (expand-file-name "elpaca/" user-emacs-directory))
(defvar elpaca-builds-directory (expand-file-name "builds/" elpaca-directory))
(when-let ((elpaca-repo (expand-file-name "repos/elpaca/" elpaca-directory))
           (elpaca-build (expand-file-name "elpaca/" elpaca-builds-directory))
           (elpaca-target (if (file-exists-p elpaca-build) elpaca-build elpaca-repo))
           (elpaca-url  "https://www.github.com/progfolio/elpaca.git")
           ((add-to-list 'load-path elpaca-target))
           ((not (file-exists-p elpaca-repo)))
           (buffer (get-buffer-create "*elpaca-bootstrap*")))
  (condition-case-unless-debug err
      (progn
        (unless (zerop (call-process "git" nil buffer t "clone" elpaca-url elpaca-repo))
          (error "%s" (list (with-current-buffer buffer (buffer-string)))))
        (byte-recompile-directory elpaca-repo 0 'force)
        (require 'elpaca)
        (elpaca-generate-autoloads "elpaca" elpaca-repo)
        (kill-buffer buffer))
    ((error)
     (delete-directory elpaca-directory 'recursive)
     (with-current-buffer buffer
       (goto-char (point-max))
       (insert (format "\n%S" err))
       (display-buffer buffer)))))
(require 'elpaca-autoloads)
(add-hook 'after-init-hook #'elpaca-process-queues)
(elpaca (elpaca :host github :repo "progfolio/elpaca" :branch "feat/use-package-support"))
(setq package-enable-at-startup nil)

(elpaca (setup :host nil :repo "https://git.sr.ht/~pkal/setup")
  (require 'setup))
(elpaca-process-queues)

(defun setup-wrap-to-install-package (body _name)
  "Wrap BODY in an `elpaca' block if necessary.
The body is wrapped in an `elpaca' block if `setup-attributes'
contains an alist with the key `elpaca'."
  (if (assq 'elpaca setup-attributes)
      `(elpaca ,(cdr (assq 'elpaca setup-attributes)) ,@(macroexp-unprogn body))
    body))

;; Add the wrapper function
(add-to-list 'setup-modifier-list #'setup-wrap-to-install-package)

(setup-define :elpaca
  (lambda (order &rest recipe)
    (push (cond
           ((eq order t) `(elpaca . ,(setup-get 'feature)))
           ((eq order nil) '(elpaca . nil))
           (`(elpaca . (,order ,@recipe))))
          setup-attributes)
    ;; If the macro wouldn't return nil, it would try to insert the result of
    ;; `push' which is the new value of the modified list. As this value usually
    ;; cannot be evaluated, it is better to return nil which the byte compiler
    ;; would optimize away anyway.
    nil)
  :documentation "Install ORDER with `elpaca'.
The ORDER can be used to deduce the feature context."
  :shorthand #'cadr)

(setup (:elpaca async))

(setup (:elpaca vibrant-ink :host github :repo "arkhan/vibrant-ink-theme")
  (if (daemonp)
      (add-hook 'after-make-frame-functions
                (lambda (frame)
                  (with-selected-frame frame (load-theme 'vibrant-ink t))))
    (load-theme 'vibrant-ink t)))

But get this:

[Riyyi]

@arkhan It is like progfolio said:

the installation will be non-blocking, so by the time the interpreter reaches the first call to the setup macro it's not guaranteed to be installed and will most likely fail.

So the very first time you start Emacs it willl fail at that point, every other time it will work fine. This is a limitation of the current configuration.

[progfolio]

So the very first time you start Emacs it willl fail at that point, every other time it will work fine.

Are you sure of this?

For simple configurations this should work:

;; -*- lexical-binding: t; -*-

(declare-function elpaca-generate-autoloads "elpaca")
(defvar elpaca-directory (expand-file-name "elpaca/" user-emacs-directory))
(defvar elpaca-builds-directory (expand-file-name "builds/" elpaca-directory))
(when-let ((elpaca-repo (expand-file-name "repos/elpaca/" elpaca-directory))
           (elpaca-build (expand-file-name "elpaca/" elpaca-builds-directory))
           (elpaca-target (if (file-exists-p elpaca-build) elpaca-build elpaca-repo))
           (elpaca-url  "https://www.github.com/progfolio/elpaca.git")
           ((add-to-list 'load-path elpaca-target))
           ((not (file-exists-p elpaca-repo)))
           (buffer (get-buffer-create "*elpaca-bootstrap*")))
  (condition-case-unless-debug err
      (progn
        (unless (zerop (call-process "git" nil buffer t "clone" elpaca-url elpaca-repo))
          (error "%s" (list (with-current-buffer buffer (buffer-string)))))
        (byte-recompile-directory elpaca-repo 0 'force)
        (require 'elpaca)
        (elpaca-generate-autoloads "elpaca" elpaca-repo)
        (kill-buffer buffer))
    ((error)
     (delete-directory elpaca-directory 'recursive)
     (with-current-buffer buffer
       (goto-char (point-max))
       (insert (format "\n%S" err))
       (display-buffer buffer)))))
(require 'elpaca-autoloads)
(add-hook 'after-init-hook #'elpaca-process-queues)
(elpaca (elpaca :host github :repo "progfolio/elpaca"))
(elpaca (setup :host nil :repo "https://git.sr.ht/~pkal/setup")
  (require 'setup)
  (defun setup-wrap-to-install-package (body _name)
    "Wrap BODY in an `elpaca' block if necessary.
The body is wrapped in an `elpaca' block if `setup-attributes'
contains an alist with the key `elpaca'."
    (if (assq 'elpaca setup-attributes)
        `(elpaca ,(cdr (assq 'elpaca setup-attributes)) ,@(macroexp-unprogn body))
      body))

  (add-to-list 'setup-modifier-list #'setup-wrap-to-install-package)

  (setup-define :elpaca
                (lambda (order &rest recipe)
                  (push (cond
                         ((eq order t) `(elpaca . ,(setup-get 'feature)))
                         ((eq order nil) '(elpaca . nil))
                         (`(elpaca . (,order ,@recipe))))
                        setup-attributes)
                  nil)
                :documentation "Install ORDER with `elpaca'.
The ORDER can be used to deduce the feature context."
                :shorthand #'cadr)

  (setup (:elpaca async))

  (setup (:elpaca vibrant-ink :host github :repo "arkhan/vibrant-ink-theme")
         (if (daemonp)
             (add-hook 'after-make-frame-functions
                       (lambda (frame)
                         (with-selected-frame frame (load-theme 'vibrant-ink t))))
           (load-theme 'vibrant-ink t)))
  (elpaca-process-queues))

@Riyyi Please test that out. If it works as expected, I would appreciate it if you would update your wiki entry to reflect a working configuration. We can improve on it later, but having one up that doesn't work or requires multiple restarts of Emacs (which is not mentioned on the wiki) isn't ideal.

[Riyyi]

So the very first time you start Emacs it willl fail at that point, every other time it will work fine.

Are you sure of this?

Yes, this is how I currently have it in my config restructure, which I haven't commited to my dotfiles repo yet until I've worked out all the kinks.

For simple configurations this should work:

This approach requires wrapping the entire config in elpaca calls, which defeat the point of the setup.el :elpaca keyword, but yes it does work on initial start. I think its best for people that want to wrap setup.el from Elpaca's point of view to just modify your elpaca-use-package macro (which I initially had issues with due to a bug in setup.el, which is now fixed), but I do agree that the wiki should mention this initial start flaw. (EDIT: I updated the wiki with a warning about this issue.)

[nasyxx]

Is there an easy blocking method now? For temporary use.

like

(elpaca xxx (do-something))

;; block
(while (not (featurep 'xxx)) 
  (sleep-for 0.1))

;; then
(elpaca yyy)

The biggest problem currently restricting my use of elpaca is that I need most of the configuration to be wrapped in (elpaca leaf)

[progfolio]

Is there an easy blocking method now?

So far I've written several experiments:

I found polling, like you mentioned above, via sit-for or accept-process-output to be inconsistent. Sometimes it works and other times the polling interval can cause the entire system to hang indefinitely. It also has the downside that the main thread/UI is blocked while polling. It's unlikely I'll pursue this strategy for a solution.

I also tried an idea where the function would intentionally throw a reader error and then use the position returned by that error to resume processing the init file once all queues up to that point finished processing. Unfortunately, the way Emacs processes the user init file didn't work with this strategy (errors are implicitly caught or explicitly hit the debugger via the --debug-init flag).

If anyone is aware of any other methods or has any ideas, I'm all ears.

The biggest problem currently restricting my use of elpaca is that I need most of the configuration to be wrapped in (elpaca leaf)

If you prefer your declarations to be top-level, the easiest solution would be to write the leaf equivalent of elpaca-use-package. Then it's a matter of substituting leaf for elpaca-leaf in your declarations.

[bestlem]

When I tried to use elpaca using use-package - the biggest issue is that elpaca puts everythiong in a background queue so the use-package can't run as its :config does not get loaded - this messes up any other package that needs to use the original.

I think it woruld work if elpaca did not do async loading.

On the first run of emacs - it is going to take time anyway so I don't see this is as an issue.

As I have not managed to get enough loaded to do a second run I can't be certain but the speed is only an issue if checking. Does elpaca load the local version and then check if the remote git is newer? If so does it replace the current one?

[progfolio]

When I tried to use elpaca using use-package - the biggest issue is that elpaca puts everything in a background queue so the use-package can't run as its :config does not get loaded - this messes up any other package that needs to use the original.

I'm not sure I understand what you mean. Could you give a simple example?

Does elpaca load the local version and then check if the remote git is newer?

Elpaca will install packages that are declared in the init file if necessary. Otherwise it just activates the versions under the builds directory. This is so one can modify a package (via editing the source files or checking out some other vc ref) without Elpaca interfering when Emacs is restarted. There are explicit commands for updating packages.

[nasyxx]

Can elpaca do this?

1. A separate el file (e.g., elpaca-pkgs.el in the ~/.emacs.d/ ) to install/load all (or bootstrap with leaf/use-package/setup.el) packages and generate autoload file and the load-path. We can use --batch to install all package or load and block the bootstrap until they are finished installing.
2. In init.el, we load the autoload and the load-path first, then we can use any of use-package, leaf, setup.el as usual.

[progfolio]

We can use --batch to install all package or load and block the bootstrap until they are finished installing.

If I understand your proposal, I'm not sure this would offer any advantage over the current situation. --batch would require blocking until everything is finished. Otherwise the Emacs process will immediately exit. If we have a reliable way to do that, there's no need split installation off into a separate file.

[nasyxx]

We could not care about the time in --batch, nor the UI block. Thus, we can do the while sit-for inside it until finished. If it hangs, we could kill it and try again.

[progfolio]

We could not care about the time in --batch, nor the UI block. Thus, we can do the while sit-for inside it until finished. If it hangs, we could kill it and try again.

I'd rather solve the problem another way than sweep it under the rug.

[librarianmage]

Has there been any update to this?

[progfolio]

The elpaca-use-package macro works fine. There isn't much benefit to more complex solutions because at best, you'll still need a level of nesting.

If you are using a different package for your init (leaf, setup.el, etc) a similar wrapping macro could be written (I will not be adding such macros to elpaca, because I don't use those packages).

If you run into specific issues while trying to convert your init file to use elpaca, please open a support ticket with the details necessary to provide help.

[librarianmage]

Using elpaca-use-package is there a way to get it to use different queues based off of the :after bits?

On Sun, Dec 18, 2022 at 4:28 PM Nicholas Vollmer @.***> wrote:

The elpaca-use-package macro works fine. There isn't much benefit to more complex solutions because at best, you'll still need a level of nesting.

If you are using a different package for your init (leaf, setup.el, etc) a similar wrapping macro could be written (I will not be adding such macros to elpaca, because I don't use those packages).

If you run into specific issues while trying to convert your init file to use elpaca, please open a support ticket with the details necessary to provide help.

— Reply to this email directly, view it on GitHub https://github.com/progfolio/elpaca/issues/7#issuecomment-1356890292, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC6P74ZOVQQD7M3UHAA45R3WN6FZ5ANCNFSM56VXLHSQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[progfolio]

Using elpaca-use-package is there a way to get it to use different queues based off of the :after bits?

No. use-package's :after keyword controls when packages are loaded. It does not control when they are installed.

[progfolio]

I think I've found a way to reliably poll for queues to finish. The elpaca-wait function will block until everything queued up to that point has finished. The elpaca-wait-poll-interval user option controls the amount of seconds to sit-for between queue checks. The log buffer should still update and the echo area should display a progress indicator, but the only user input that will register is key-board-quit (C-g). Quitting will break out of the polling loop, but does not kill any running sub-processes at this point.

I've also included use-package keyword support via elpaca-use-package. I'd appreciate feedback on the feat/new-elpaca-wait feature branch if anyone has the time.

Known limitations:

• Don't nest your use-package declarations. This isn't a common approach, but I do see it some times. Nested declarations are currently queued, but they don't get processed. I had one such instance in my init file and I haven't been able to get it working, even by manually calling elpaca-process-queues after the nested declarations. I'll look into it, but it's low priority.

• elpaca-use-package.el does not get byte-compiled. This is because I wrote it to directly require and depend on use-package as if it were an external, stand-alone package. I'm not sure how I'd like to deal with this yet. I may change the implementation, or I may change Elpaca's default order to exclude it. Then Elpaca could handle installing use-package when necessary without the user installing it. This is low priority, too, until the feature gets adequate testing.

• I've only tested this on my laptop. I'm weary of inconsistencies with polling, but I think this is worth testing. There does not seem to be any significant performance difference due to polling. I'd like to find an optimal default value for elpaca-wait-poll-interval, if there is one.

The following init file template demonstrates how to test these new features:

;; Example elpaca-wait + elpaca-use-package-mode configuration -*- lexical-binding: t; -*-

(defvar elpaca-directory (expand-file-name "elpaca/" user-emacs-directory))
(defvar elpaca-builds-directory (expand-file-name "builds/" elpaca-directory))
(defvar elpaca-order '(elpaca :repo "https://github.com/progfolio/elpaca.git"
                              :ref "feat/elpaca-wait"
                              :build (:not elpaca--activate-package)))
(when-let ((repo  (expand-file-name "repos/elpaca/" elpaca-directory))
           (build (expand-file-name "elpaca/" elpaca-builds-directory))
           (order (cdr elpaca-order))
           ((add-to-list 'load-path (if (file-exists-p build) build repo)))
           ((not (file-exists-p repo))))
  (condition-case-unless-debug err
      (if-let ((buffer (pop-to-buffer-same-window "*elpaca-bootstrap*"))
               ((zerop (call-process "git" nil buffer t "clone"
                                     (plist-get order :repo) repo)))
               (default-directory repo)
               ((zerop (call-process "git" nil buffer t "checkout"
                                     (or (plist-get order :ref) "--")))))
          (progn
            (byte-recompile-directory repo 0 'force)
            (require 'elpaca)
            (and (fboundp 'elpaca-generate-autoloads)
                 (elpaca-generate-autoloads "elpaca" repo))
            (kill-buffer buffer))
        (error "%s" (with-current-buffer buffer (buffer-string))))
    ((error)
     (warn "%s" err)
     (delete-directory repo 'recursive))))
(require 'elpaca-autoloads)
(add-hook 'after-init-hook #'elpaca-process-queues)
(elpaca `(,@elpaca-order))

;; Install use-package
(elpaca use-package
  ;; Enable :eplaca keyword support
  (elpaca-use-package-mode)
  ;; Assume :elpaca t for packages unless otherwise specified.
  (setq elpaca-use-package-by-default t))

;; Process use-package's queue. This will block!
(elpaca-wait)

;; expands to:
;;
;;(elpaca doct
;;  (require 'doct nil nil))
(use-package doct)

;; Don't install anything, defer BODY.
;; expands to:
;;
;;(elpaca nil
;; (require 'project nil nil))
(use-package project :elpaca nil)

;; Local Variables:
;; no-byte-compile: t
;; End:

As always, testing, questions, ideas are all appreciated.

[bestlem]

Well simple file worked - so I will attempt my full init.

However one issue I had to delete all the elpaca downloaded things including elpaca as on first run of the new bootstrap it found the old elpaca and so could not find elpaca-wait.

So some form of checking for version of elpaca might be needed.

[progfolio]

Well simple file worked - so I will attempt my full init.

Thanks for testing. I found a bug, myself, where the queues are all completed, but the queue after an elpaca-wait is not being marked completed. I'll work on figuring that out today.

What version of Emacs are you running? So far I've successfully bootstrapped with the demo file in 27.2, 28.2 and 30.0.5 as of my build from 12/18.

So some form of checking for version of elpaca might be needed.

I'll have to think on this. Elpaca typically doesn't change what's on disk unless you ask it to. That's enables one to develop elisp packages easily. It will probably be a matter of adding an argument to elpaca-rebuild which rebuilds "from scratch", checking out the ref prior to rebuilding.

[bestlem]

Ok my attempt at my init failed. This is on 30.0.5

I think the problem is that I use functions from the package dash in my init,

Debugging is more difficult as debug-on-error seems to get set to nil and/or the error is in the background as the function from dash is in a function.

[progfolio]

Ok my attempt at my init failed. This is on 30.0.5

I think the problem is that I use functions from the package dash in my init,

Debugging is more difficult as debug-on-error seems to get set to nil and/or the error is in the background as the function from dash is in a function.

Please share your init file and I can see if I can get it working on my end.

[bestlem]

A cut down version with the same error. But any use of dash would show this.

;; -*- lexical-binding:t;coding: utf-8 -*-
(defvar elpaca-directory (expand-file-name "elpaca/" mwb-emacs-work-dir))
(defvar elpaca-builds-directory (expand-file-name (concat "builds-" emacs-version) elpaca-directory))
(defvar elpaca-order '(elpaca :repo "https://github.com/progfolio/elpaca.git"
                       :ref "feat/new-elpaca-wait"
                       :build (:not elpaca--activate-package)))

(when-let ((repo  (expand-file-name "repos/elpaca/" elpaca-directory))
           (build (expand-file-name "elpaca/" elpaca-builds-directory))
           (order (cdr elpaca-order))
           ((add-to-list 'load-path (if (file-exists-p build) build repo)))
           ((not (file-exists-p repo))))
  (condition-case-unless-debug err
      (if-let ((buffer (pop-to-buffer-same-window "*elpaca-bootstrap*"))
               ((zerop (call-process "git" nil buffer t "clone"
                                     (plist-get order :repo) repo)))
               (default-directory repo)
               ((zerop (call-process "git" nil buffer t "checkout"
                                     (or (plist-get order :ref) "--")))))
          (progn
            (byte-recompile-directory repo 0 'force)
            (require 'elpaca)
            (and (fboundp 'elpaca-generate-autoloads)
                 (elpaca-generate-autoloads "elpaca" repo))
            (kill-buffer buffer))
        (error "%s" (with-current-buffer buffer (buffer-string))))
    ((error)
     (warn "%s" err)
     (delete-directory repo 'recursive))))
(require 'elpaca-autoloads)
(add-hook 'after-init-hook #'elpaca-process-queues)
(elpaca `(,@elpaca-order))

;; Install use-package
(elpaca use-package
        ;; Enable :eplaca keyword support
        (elpaca-use-package-mode)
        ;; Assume :elpaca t for packages unless otherwise specified.
        (setq elpaca-use-package-by-default t))

;; Process use-package's queue. This will block!
(elpaca-wait)

                                        ; install use-package
;; (eval-when-compile
;;   (setq use-package-enable-imenu-support t)
;;   (require 'use-package)
;;   (setq bind-key-describe-special-forms t)
;;   (setq use-package-always-defer t)

;;   (setq byte-compile-current-buffer t)
;;   (if init-file-debug
;;       (setq use-package-verbose t
;;             use-package-expand-minimally nil
;;             use-package-compute-statistics t)
;;     (setq use-package-verbose nil
;;           use-package-expand-minimally t)))

(use-package dash
  :demand
  :config
  (global-dash-fontify-mode))

(defun mwb-insert-before-element (find-element new-element list)
  "Find FIND-ELEMENT and then insert NEW-ELEMENT before it in LIST."
  (let ((i (-elem-index find-element list)))
    (-insert-at i new-element list)))

(defun use-package-normalize-mwb-paths (label arg &optional recursed)
  "Normalize a list of filesystem paths."
  (cond
   ((and arg (or (use-package-non-nil-symbolp arg) (functionp arg)))
    (let ((value (use-package-normalize-value label arg)))
      (use-package-normalize-paths label (eval value))))
   ((stringp arg)
    (let ((path (if (file-name-absolute-p arg)
                    arg
                  (mwb-user-emacs-file arg))))
      (list path)))
   ((and (not recursed) (listp arg) (listp (cdr arg)))
    (mapcar #'(lambda (x)
                (car (use-package-normalize-paths label x t))) arg))
   (t
    (use-package-error
     (concat label " wants a directory path, or list of paths")))))

;;;; :mwb-load-path

(defun use-package-normalize/:mwb-load-path (_name keyword args)
  (use-package-as-one (symbol-name keyword) args
    #'use-package-normalize-mwb-paths))

(defun use-package-handler/:mwb-load-path (name keyword arg rest state)
  (use-package-handler/:load-path name keyword arg rest state))

(setq use-package-keywords
      (mwb-insert-before-element :load-path :mwb-load-path  use-package-keywords)
      )

;; Local Variables:
;; no-byte-compile: t
;; End:

[progfolio]

@bestlem Once elpaca-use-package-mode is enabled, use-package declarations are queued by Elpaca. If you need a queue to be processed (because non-deferred forms in your init file rely on it), you can add another (elpaca-wait) call. Try adding one after dash's use-package declaration.

[bestlem]

Current state is that I have one failure paredit with error fatal: dumb http transport does not support shallow capabilities 01.530858 which I suspect is an elpaca bug not related to the use-package integration,

I have had packages with links to the wrong package and the autoload file having the contents of the mislinked package.

When I try this package in a simple test case (use-package guess-style :commands guess-style-guess-all guess-style-info-mode) I get the error elpaca--clone: (wrong-type-argument stringp nil)

So something is broken here as I suspect I should get that in my main init and not the

I also have a package that says it is blocked Waiting on monorepo. Thgat repo has been loaded and code is running from it.

Another problem is that the :commands keyword in use-package is not working ie the command is not loaded. It worked in a simple test even with paredit attempted before it.

I suspect the blocked package meand that the queue has notr finished and so the hooks don't run.

[progfolio]

Current state is that I have one failure paredit with error fatal: dumb http transport does not support shallow capabilities 01.530858 which I suspect is an elpaca bug not related to the use-package integration,

Elpaca defaults to shallow clones, the server offering paredit does not support that. Specifying :depth nil in the package recipe should request a full clone of the repository. The :depth recipe keyword is documented here:

https://github.com/progfolio/elpaca/blob/master/doc/manual.md#recipes

I have an idea for a feature which would offer solutions to common package failures under certain circumstances. It's not a high priority at the moment and I still have to experiment with the design, though.

I have had packages with links to the wrong package and the autoload file having the contents of the mislinked package.

I also have a package that says it is blocked Waiting on monorepo. Thgat repo has been loaded and code is running from it.

These sound like a bugs. If you can reproduce these, please fill out a separate issue for each and I will take a look into fixing them.

Another problem is that the :commands keyword in use-package is not working ie the command is not loaded. It worked in a simple test even with paredit attempted before it.I suspect the blocked package meand that the queue has notr finished and so the hooks don't run.

If the order fails, it's BODY forms are not run. So, with elpaca-use-package-mode enabled, the body of the use-package declaration will not run if the installation/build fails.

Thanks for taking the time to test.

[reuleaux]

I have finally given elpaca another try, and I can say that it works for me with leaf ! :-) (I have only just ported my complete config from straight.el to elpaca - thus I am not really experienced with elpaca yet, but experience will come, of course, over time).

I am attaching a relatively minimal init.el below, with some basic settings: vertico, orderless, consult etc. -

Allow me some comments beforehand:

First, when I started (again now), I thought, that I just use the - longer to type - detailed recipes - requiring leaf explicitly - as for example for zig-mode:

(elpaca zig-mode
  (require 'zig-mode)
  (leaf nil ,(elpaca-first leaf)
        :defvar zig-format-on-save
        :setq ((zig-format-on-save . nil))))

having had some bad experiences with porting to the elpaca-usepackage macro to leaf earlier (in August i.e.) (At least, I would be in a leaf block then - and be able to use leaf config setting (:setq, :defvar, ...) as in the code above.)

Now apparently - this elpaca-usepackage has changed since August (is more general - applicable to leaf as well - as I understand), and thus the current version was easy to adapt to leaf (I have just literally replaced usepackage with leaf):

;;;###autoload
(defmacro elpaca-leaf (order &rest body)
  "Execute BODY in `leaf' declaration after ORDER is finished.
If the :disabled keyword is present in body, the package is completely ignored.
This happens regardless of the value associated with :disabled.
The expansion is a string indicating the package has been disabled."
  (declare (indent 1))
  (if (memq :disabled body)
      (format "%S :disabled by elpaca-leaf" order)
    (let ((o order))
      (when-let ((ensure (cl-position :ensure body)))
        (setq o (if (null (nth (1+ ensure) body)) nil order)
              body (append (cl-subseq body 0 ensure)
                           (cl-subseq body (+ ensure 2)))))
      `(elpaca ,o (leaf
                    ,(if-let (((memq (car-safe order) '(quote \`)))
                              (feature (flatten-tree order)))
                         (cadr feature)
                       (elpaca--first order))
                    ,@body)))))

Now - while I haven't even tried to understand, what's going on this macro in detail, this just works and makes things easier for me - the above zig recipe with this macro is shorter, of course:

(elpaca-leaf zig-mode
    :defvar zig-format-on-save
    :setq
    ((zig-format-on-save . nil)))

Now, two more comments, before I show my minimal/complete init.el:

First I make use of a small macro msg in file _debug.el as follows,

(defmacro msg (&rest args)
  (when (and (boundp 'debug-level) (eq debug-level :debug))
    `(message (format ,@args))))
(provide '_debug)

which allows me to output (or not) some messages - depending on the debug level I set, this is easy enough to understand - and I have just left it in here - adjust the load-path in init.el below according to where this _debug.el file is found (in my case):

(add-to-list 'load-path "~rx/elpaca")

Second: personally I do byte-compile my init.el once in a while - if only to see warnings/errors, if any. - I have since learned, that this is not recommended any more - cf. the now closed issue:

https://github.com/progfolio/elpaca/issues/9

And indeed, if I do this now:

emacs --batch -f batch-byte-compile init.el

elpaca even gets in my way with an error:

Debugger entered--Lisp error: (void-function elpaca--queue)
  elpaca--queue((elpaca :repo "https://github.com/progfolio/elpaca.git" :ref nil :build (:not elpaca--activate-package)))
  byte-code("\304\305\10\"\211\205\f\0\304\306\11\"\211\205\22\0\nA\211\205$\0\307\310\311\4!\203\"\0\3\202#\0\4\"\211\205,\0\311\4!?\211\203\252\0\3121..." [elpaca-directory elpaca-builds-directory elpaca-order default-directory expand-file-name "repos/elpaca/" "elpaca/" add-to-list load-path file-exists-p (debug error) pop-to-buffer-same-window "*elpaca-bootstrap*" call-process "git" nil t "clone" plist-get :repo 0 "checkout" :ref "--" byte-recompile-directory force require elpaca fboundp elpaca-generate-autoloads "elpaca" kill-buffer error "%s" buffer-string warn delete-directory recursive elpaca-autoloads add-hook after-init-hook elpaca-process-queues elpaca--queue] 17)
  load("/home/rx/.emacs.d/init" noerror nomessage)
  startup--load-user-init-file(#f(compiled-function () #<bytecode 0x14f594fae0bf006a>) #f(compiled-function () #<bytecode -0x1f3c686ddc0cdc35>) t)
  command-line()
  normal-top-level()

and I will not even try to resolve this issue, I just remove the byte compiled init.el:

rm -f init.elc

I have nevertheless left my code - as it is: most code wrapped in eval-and-compile or in eval-when-compile i.e. - this just reflects how I work today (how I get to see debug messages etc) - I will be happy to remove these eval-and-compile wrappers in the future.

I do see - when I start (elpaca-configured) emacs a second time - that there is some active native compiling going on (cf. the buffer Async-native-compile-log - and I am sure, this native code is faster, and the way to go in the future - but I yet have to wrap my head around this native compilation stuff).

Thus - with many eval-and-compile wrapped blocks (remove them, if you want an ever simpler init.el), without further ado, here comes my minimal elapaca/leaf init.el:

;; -*- lexical-binding: t  -*-

(eval-and-compile
  (add-to-list 'load-path "~rx/elpaca")
  (require '_debug)
  (setq debug-on-error t)
  (defvar debugging-on t)
  (cond
   ((daemonp)
    (defvar debug-level :debug))
   ;; turn on w/ em -compile -debug
   (debugging-on (defvar debug-level :debug))
   (t (defvar debug-level :other))))

(eval-and-compile
  (defvar elpaca-directory (expand-file-name "elpaca/" user-emacs-directory))
  (defvar elpaca-builds-directory (expand-file-name "builds/" elpaca-directory))
  (defvar elpaca-order '(elpaca :repo "https://github.com/progfolio/elpaca.git"
                                :ref nil
                                :build (:not elpaca--activate-package)))
  (when-let ((repo  (expand-file-name "repos/elpaca/" elpaca-directory))
             (build (expand-file-name "elpaca/" elpaca-builds-directory))
             (order (cdr elpaca-order))
             ((add-to-list 'load-path (if (file-exists-p build) build repo)))
             ((not (file-exists-p repo))))
    (condition-case-unless-debug err
        (if-let ((buffer (pop-to-buffer-same-window "*elpaca-bootstrap*"))
                 ((zerop (call-process "git" nil buffer t "clone"
                                       (plist-get order :repo) repo)))
                 (default-directory repo)
                 ((zerop (call-process "git" nil buffer t "checkout"
                                       (or (plist-get order :ref) "--")))))
            (progn
              (byte-recompile-directory repo 0 'force)
              (require 'elpaca)
              (and (fboundp 'elpaca-generate-autoloads)
                   (elpaca-generate-autoloads "elpaca" repo))
              (kill-buffer buffer))
          (error "%s" (with-current-buffer buffer (buffer-string))))
      ((error)
       (warn "%s" err)
       (delete-directory repo 'recursive))))
  (require 'elpaca-autoloads)
  (add-hook 'after-init-hook #'elpaca-process-queues)
  (elpaca `(,@elpaca-order)))

;; copied/adapted in Jan' 2023
;;;###autoload
(defmacro elpaca-leaf (order &rest body)
  "Execute BODY in `leaf' declaration after ORDER is finished.
If the :disabled keyword is present in body, the package is completely ignored.
This happens regardless of the value associated with :disabled.
The expansion is a string indicating the package has been disabled."
  (declare (indent 1))
  (if (memq :disabled body)
      (format "%S :disabled by elpaca-leaf" order)
    (let ((o order))
      (when-let ((ensure (cl-position :ensure body)))
        (setq o (if (null (nth (1+ ensure) body)) nil order)
              body (append (cl-subseq body 0 ensure)
                           (cl-subseq body (+ ensure 2)))))
      `(elpaca ,o (leaf
                   ,(if-let (((memq (car-safe order) '(quote \`)))
                             (feature (flatten-tree order)))
                        (cadr feature)
                      (elpaca--first order))
                   ,@body)))))

(elpaca leaf (require 'leaf))
(elpaca leaf-keywords (require 'leaf-keywords))

(elpaca-leaf nil
             :init (leaf leaf-keywords
                         :init (leaf-keywords-init)))
(eval-when-compile (msg "leaf done."))

(eval-and-compile (elpaca-leaf vertico :init (vertico-mode)))
(eval-when-compile (msg "vertico done."))
(eval-and-compile (elpaca-leaf nil :init (savehist-mode)))
(eval-when-compile (msg "savehist done."))
;; A few more useful configurations...
;; cf https://github.com/minad/vertico

(elpaca-leaf nil
             :init
             ;; Do not allow the cursor in the minibuffer prompt
             (setq minibuffer-prompt-properties
                   '(read-only t cursor-intangible t face minibuffer-prompt))
             (add-hook 'minibuffer-setup-hook #'cursor-intangible-mode)

             ;; Emacs 28: Hide commands in M-x which do not work in the current mode.
             ;; Vertico commands are hidden in normal buffers.
             ;; (setq read-extended-command-predicate
             ;;       #'command-completion-default-include-p)

             ;; Enable recursive minibuffers
             (setq enable-recursive-minibuffers t))
(eval-when-compile (msg "additional emacs config done."))

(eval-and-compile
  (elpaca-leaf orderless
               :init
               (setq completion-styles '(orderless)
                     completion-category-defaults nil
                     completion-category-overrides '((file (styles partial-completion))))))
(eval-when-compile (msg "orderless done."))

(eval-and-compile
  (elpaca-leaf consult
               :require t
               :bind ((("C-s" . consult-line)
                       ("C-S-s" . isearch-forward)
                       ("C-S-r" . isearch-backward)
                       ("M-y" . consult-yank-from-kill-ring)
                       ("C-x b" . consult-buffer)))))
(eval-when-compile (msg "consult done."))

[progfolio]

@reuleaux Thanks for the example. I applied some formatting to your comment (reformatted dangling parens, enabled proper elisp syntax highlighting in code blocks by adding emacs-lisp as the language at the start of the code fence instead of el).

I would remove the usage of eval-when-compile considering the file isn't meant to be byte-compiled. That's likely the source of that error you were seeing about elpaca--queue. You may also be able to do away with your msg macro. Everything your are logging (and more) is logged by Elpaca. You can view this via the elpaca-log and the elpaca-status commands.

[reuleaux]

OK, thanks a lot for your feedback (and your formatting) - And I will, of course, continue to improve/refine my code (and this includes: getting rid of my eval-and-compile / eval-when-compile wrappers in the near future, as you suggested). - This is all just work in progress for me, after all.

I guess, the important message at this point is: yes, it is possible to use elpaca with with leaf - either in the detailed/verbose manner, or with the elpaca-leaf macro - both shown above.

[maddovr]

For setup.el, I checked out the feat/elpaca-wait branch but now it fails when updating. Also the snippet posted in the emacswiki prevents multiple elpaca invocation the same body, contrary to the straight.el snippet, that makes configuring packages that are all used in the same context but with a specific load-order (think all-the-icons and all its modules) cumbersome. Are there any news on this issue? Also, as the wiki reports, the script fails to bootstrap on first run, here's the relevant link https://www.emacswiki.org/emacs/SetupEl#h5o-19

[progfolio]

For setup.el, I checked out the feat/elpaca-wait branch but now it fails when updating.

Please open a separate issue and provide the requested information so I can help.

Also the snippet posted in the emacswiki prevents multiple elpaca invocation the same body, contrary to the straight.el snippet, that makes configuring packages that are all used in the same context but with a specific load-order (think all-the-icons and all its modules) cumbersome. Are there any news on this issue? Also, as the wiki reports, the script fails to bootstrap on first run, here's the relevant link https://www.emacswiki.org/emacs/SetupEl#h5o-19

I did not author that wiki text. I have removed it because I do not recommend setting up Elpaca that way.

[Riyyi]

@maddovr Im the author of that wiki entry and I no longer recommend setting it up that way either. I only needed to because of a bug in setup.el which has been fixed. Everything works for me by modifying the elpaca-use-package macro.

[progfolio]

The "feat/elpaca-wait" branch has been merged into master. This includes the blocking elpaca-wait function and use-package support via elpaca-use-package-mode. The elpaca-use-package macro has been removed as well (I may bring it back, as not all setups require elpaca-wait, and it should be marginally faster than blocking). As always, testing is appreciated. Please open a new issue with the requested information when reporting bugs with these features.

[Riyyi]

@progfolio Im a little bit confused; you removed the setup.el wiki entry due to the reliance on a blocking call, but you added that functionality with elpaca-wait now. The alternative, the elpaca-use-package macro (with use-package replaced to setup) was also removed. So what is the recommended way to use setup.el with Elpaca now?

[progfolio]

you removed the setup.el wiki entry due to the reliance on a blocking call

The code on the wiki page did not block. It threw an error after setup.el was queued. The user was supposed to ignore this failure, wait for setup.el to install, then restart Emacs, no? You mention as much here: https://github.com/progfolio/elpaca/issues/7#issuecomment-1253297761

Now that elpaca-wait has been installed, the wiki entry could be re-written to reflect that. A proper setup would be along the lines of:

1. Bootstrap Elpaca
2. Install setup.el via Elpaca. Code to extend setup.el to work with Elpaca should go in the BODY of the declaration.
3. Call elpaca-wait to block until step 2 is finished.
4. Use setup.el with Elpaca extensions at top-level.

A template should be available here (sans setup.el specific code, because I am not familiar enough with it to extend it myself): https://github.com/progfolio/elpaca/issues/58#issuecomment-1411036871 I have no objections to a working configuration being added to the wiki. I just did not want one which relied on broken behavior being recommended.

[Riyyi]

The code on the wiki page did not block. It threw an error after setup.el was queued. The user was supposed to ignore this failure, wait for setup.el to install, then restart Emacs, no?

Yes, it did a (elpaca-process-queues) instead of (elpaca-wait) as it hadnt been merged yet. Otherwise the wiki entry did the steps you listed, the only limitation was that it could only do a single Elpaca invocation in the same body.

Personally I like the macro method better, as its less complex and syntactically a little better.

(elpaca-setup org
  ..)

(elpaca nil (setup org
  ..))

versus

(setup (:elpaca org)
  ..)

(setup org
  (:elpaca nil)
  ..)

[progfolio]

Yes, it did a (elpaca-process-queues) instead of (elpaca-wait)

elpaca-process-queues kicks off async processes. It does not block the interpreter until the queues are processed. If that were the case, there would be no need for elpaca-wait.

Feel free to evaluate the following test in your *scratch* buffer to confirm:

(elpaca-test
  :early-init
  (setq inhibit-automatic-native-compilation t
        inhibit-splash-screen t
        package-enable-at-startup nil)
  :init
  (add-hook 'elpaca-after-init-hook (lambda () (pop-to-buffer "*Messages*")))
  (elpaca nil (message "Configuring setup.el"))
  (elpaca-process-queues)
  (message "Using setup.el"))

You should see "Using setup.el" messaged prior to "Configuring setup.el". That explains the error you mentioned here:

https://github.com/progfolio/elpaca/issues/7#issuecomment-1253297761

the very first time you start Emacs it willl fail at that point, every other time it will work fine.

That method will fail any time setup.el needs to be installed or re-built during init.

Personally I like the macro method better, as its less complex

I think a wrapping macro works fine. I'll likely continue using one in my init file for use-package. However, others wanted a way to block, so elpaca-wait should cover that case.

and syntactically a little better.

I can't comment on the examples you've given because I don't use or know enough about setup.el. I'll leave authoring/managing macros for other init configuration packages to interested users.

In terms of the syntatic difference for use-package, both the wrapping macro and elpaca-use-package-mode require minimal changes to one's init.