Let user choose whether development tools are fetched from an archive or provided via the load-path

[suhail-singh]

When building and testing a package in certain environments without network access (e.g. when building packages in the Guix package manager), one may wish to manually provide certain dependencies like test-frameworks.

For instance, the ox-tufte package uses both ert and buttercup test frameworks. When eldev test for this package is invoked where the buttercup dependency is available in the EMACSLOADPATH, Eldev still tries to fetch buttercup from melpa. When this action is performed in the Guix build environment (where there is no network access) the process fails. For completeness, even eldev test-ert fails for this package because eldev-test-framework specifies '(ert buttercup) and so an attempt to fetch buttercup is made regardless. When buttercup is removed from eldev-test-framework, then eldev test-ert works.

The desired workflow is one where the user ensures that the development tool is available in the load-path (by way of EMACSLOADPATH or otherwise), and passes a commandline argument to eldev instructing it to not fetch the development tool from the archive(s). Is this possible to do in the current version of Eldev? If so, could you please point me in the right direction? If not, could this feature please be added?

~~On a related note, is there a way to work around the current limitation (should it not simply be a result of my lack of awareness) by adding some code to the project-specific Eldev file?~~

[suhail-singh]

Answering myself:

is there a way to work around the current limitation

The naive workaround is to not use Eldev. For instance, in the case of buttercup tests, the below works.

emacs --batch -L . -f buttercup-run-discover

[doublep]

This looks pretty similar to #64. As suggested there, you could either use global option --external-deps to take preinstalled packages (e.g. from the normal Emacs installation) or local dependencies if you want to make something not fetched over internet selectively. Would that be enough in your usecase?

[suhail-singh]

Would that be enough in your usecase?

I'm not sure, but I am leaning towards "might be sufficient, but inconvenient".

In the specific instance of building packages on Guix, which I'm considering, the invariance that's maintained is that the dependencies will be available in the directories (note plural) listed in $EMACSLOADPATH. The environment variable EMACSLOADPATH is dynamically set at build time - the developer specifies the package dependencies and Guix ensures that the location of said dependencies is listed in $EMACSLOADPATH. As such, at build time, when emacs is started, load-path gets appropriately set and Emacs is able to locate the packages. However, it seems, this isn't quite sufficient for Eldev.

When using --external-deps=$foo, IIUC the expectation is for the dependencies (say, $dep1 and $dep2) to be both "installed" in $foo. When that's not the case, Eldev exits with Dependency ‘$dep1’ is not installed in ‘$foo’. In the case of Guix, $dep1 and $dep2 are expected to be in distinct directories so this doesn't work.

If, however, local dependencies are to be used, then at build time the user would have to generate an Eldev-local file (or pass the form via --setup) with an eldev-use-local-dependency invocation for each dependency ($dep1 and $dep2) with the appropriate entries from $EMACSLOADPATH. And while that's doable, it isn't convenient.

For completeness, it seems by (mis-)using local dependencies it might be possible to write a generic Eldev-local file (or pass the form via --setup) that tries to add every entry in $EMACSLOADPATH as a local dependency. Something like:

(dolist (entry (split-string (getenv "EMACSLOADPATH") ":" t))
  (ignore-errors ;; Eldev errors if local dependency is the package being built
    (eldev-use-local-dependency entry)))

I don't know if doing this can lead to other, possibly subtle, issues elsewhere. And if it cannot and this is, indeed, the recommended approach, could the above option be made available via a commandline flag for convenience?

[doublep]

might be sufficient, but inconvenient

Well, it would also be very inconvenient for Eldev to try and support yet another way of looking up dependencies. Please see the comment in #64 that explains why supporting load-path is difficult and goes against how dependencies are looked up by Eldev.

The environment variable EMACSLOADPATH is dynamically set at build time - the developer specifies the package dependencies and Guix ensures that the location of said dependencies is listed in $EMACSLOADPATH.

Who does that? Do you have any control over the code that sets the environment variable (and are you thus able to adjust it), or do you have to accept it like it is?

(dolist (entry (split-string (getenv "EMACSLOADPATH") ":" t)) (ignore-errors ;; Eldev errors if local dependency is the package being built (eldev-use-local-dependency entry)))

Well, that looks like a sane piece of code. Using full-fledged Elisp in setup files is supposed to be Eldev feature. Instead of using ignore-errors, I would compare entry against eldev-project-dir using file-equal-p to avoid hiding other potential errors.

As an additional minor advantage, you can pass whatever loading-mode parameter to eldev-use-local-dependency.

[suhail-singh]

Well, it would also be very inconvenient for Eldev to try and support yet another way of looking up dependencies.

Perhaps. But it's worth noting that the specific "way of looking up dependencies" is one already supported by Emacs.

Do you have any control over the code that sets the environment variable (and are you thus able to adjust it), or do you have to accept it like it is?

The EMACSLOADPATH variable is implicitly set. The packager will note, among other things, the build-time dependencies, and the build system ensures that those are installed and available in EMACSLOADPATH for any code running during the build phase (e.g. tests). While I don't have personal experience with it, I believe it would be possible to modify EMACSLOADPATH during the build phase if one so desired. However, I don't believe that that'd be useful. The code run during the build phase doesn't use the host OS's Emacs nor the host Emacs configuration. In fact the environment within which the build-phase code is evaluated doesn't have network access. The EMACSLOADPATH contains all and only those dependencies that are noted in the "Guix package definition".

Btw, I tried using --setup and, in the specific instance where $dep1 is a development tool, the below doesn't work.

eldev -S '(dolist (entry (split-string (getenv "EMACSLOADPATH") ":" t)) (ignore-errors (eldev-use-local-dependency entry)))' test

~~However, having the form above be in Eldev-local and then invoking eldev test works.~~ Am I doing something wrong when passing -S? ~~Or is it that --setup evaluates the form too late and thus this needs to be specified via Eldev-local?~~

EDIT: Regarding above, it seems the error was due to a lack of --external-deps. Once --external-deps is set to a directory with write access things work.

Instead of using ignore-errors, I would compare entry against eldev-project-dir using file-equal-p to avoid hiding other potential errors.

FWIW, it wouldn't have occurred to me that the above ought to be preferred over ignore-errors. I.e., others like me would probably benefit from this "mode of Eldev execution" being accessible via a commandline flag.

As an additional minor advantage, you can pass whatever loading-mode parameter to eldev-use-local-dependency.

When building in Guix, the only loading-mode that would make sense would be as-is (the default). When using Guix to package for other environments there may be utility in tweaking the loading-mode.

[suhail-singh]

Turns out it's not so straightforward to obviate the use of ignore-errors. Specifically, another error that eldev-use-local-dependency can throw is when the directory in question contains "No .el files with package headers".

For Guix, it seems the right incantation is something like

eldev --external-deps=./.eldev/ -S '(dolist (entry (split-string (getenv "EMACSLOADPATH") ":" t)) (ignore-errors (eldev-use-local-dependency entry)))' -dtTC test

[doublep]

In 1.9 there will be --disable-dependencies (alias: --use-emacsloadpath; no short variant). Please test if that's what you need. As currently implemented, you need to include everything in EMACSLOADPATH, even the current directory.

[suhail-singh]

If the current directory is the directory where the package source resides (that uses Eldev and whose tests we are trying to run), would that still need to be added to EMACSLOADPATH? If so, that may be a little awkward to use, but I'll know for certain once I test the upcoming Eldev version.

Though, if the feature in question is already on master and is "done" from your perspective, I could report on it without waiting on the release. Please let me know.

[doublep]

Yes, it is committed. You can also test it via standard installation after running

$ eldev  --unstable upgrade-self

You can later switch back to the released version by using

$ eldev upgrade-self -d

Paul

On Sat, 17 Feb 2024 at 00:02, suhail-singh @.***> wrote:

If the current directory is the directory where the package source resides (that uses Eldev and whose tests we are trying to run), would that still need to be added to EMACSLOADPATH? If so, that may be a little awkward to use, but I'll know for certain once I test the upcoming Eldev version.

Though, if the feature in question is already on master and is "done" from your perspective, I could report on it without waiting on the release. Please let me know.

— Reply to this email directly, view it on GitHub https://github.com/emacs-eldev/eldev/issues/99#issuecomment-1949459081, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYPBJR73RRRZNYNEXRKZLYT7QRRAVCNFSM6AAAAABCLE5D7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBZGQ2TSMBYGE . You are receiving this because you commented.Message ID: @.***>

[suhail-singh]

I was not able to get the build to work on commit 92de7f9 using --use-emacsloadpath.

When using the below comandline invocation to build a package (say, pkg-foo):

eldev --use-emacsloadpath -dtTC test

The following exception is raised:

...
[00:00.035]  Debugger entered--Lisp error: (error "Unable to unload package ?pkg-foo?")
               signal(error ("Unable to unload package ?pkg-foo?"))
               error("Unable to unload package `%s'" pkg-foo)
               eldev--unload-package(pkg-foo)
               eldev--do-build((":autoloads"))
               eldev-build(":autoloads")
               #f(compiled-function (type additional-sets) #<bytecode 0x5a63cfd5ffc4b40>)(t (test))
               run-hook-with-args(#f(compiled-function (type additional-sets) #<bytecode 0x5a63cfd5ffc4b40>) t (test))
               eldev-load-project-dependencies(test)
               eldev--do-test((ert buttercup) nil)
               eldev-test()
               apply(eldev-test nil)
               eldev--execute-command(("test"))
               eldev-cli(("--use-emacsloadpath" "-dtTC" "test"))
               (kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil)))
               eval((kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil))) t)
               command-line-1(("--execute" "(let ((eldev--emacs-version (format \"%s.%s\" emacs-major-version emacs-minor-version))\n      (eldev--dir           (getenv \"ELDEV_DIR\"))\n      ;; This is intentional.  First, this is in case ELDEV_LOCAL is\n      ;; defined, second, this is just Eldev default for packages.\n      (load-prefer-newer    t))\n  ;; Setting `debug-on-error' would be useful, but it can break many\n  ;; `package-*' functions, since those use `with-demoted-errors' and\n  ;; so `condition-case-unless-debug'.\n  (unless (and (fboundp 'version<=) (version<= \"24.1\" eldev--emacs-version))\n    (error \"Eldev requires Emacs 24.1 or newer\"))\n  (setf package-user-dir\n        (expand-file-name \"bootstrap\"\n                          (expand-file-name eldev--emacs-version\n                                            (if (> (length eldev--dir) 0)\n                                                eldev--dir\n                                              (if (file-directory-p \"~/.eldev\")\n                                                  \"~/.eldev\"\n                                                ;; Duplicating not-yet-available code from `eldev-xdg-cache-home'.\n                                                (expand-file-name \"eldev\"\n                                                                  (let ((eldev--xdg-cache-dir (getenv \"XDG_CACHE_HOME\")))\n                                                                    (if (and eldev--xdg-cache-dir (file-name-absolute-p eldev--xdg-cache-dir))\n                                                                        eldev--xdg-cache-dir\n                                                                      \"~/.cache\")))))))\n        package-directory-list nil\n        package-archives       nil)\n  (require 'package)\n  (package-initialize t)\n  (let ((package-archives '((\"melpa-stable\" . \"http://stable.melpa.org/packages/\")))\n        (archive-name      \"MELPA Stable\")\n        (inhibit-message  t)\n        (eldev-local      (getenv \"ELDEV_LOCAL\"))\n        eldev-pkg\n        requirements)\n    (unless (= (length eldev-local) 0)\n      (if (string-prefix-p \":pa:\" eldev-local)\n          (setf package-archives `((\"bootstrap-pa\" . ,(file-name-as-directory (substring eldev-local (length \":pa:\")))))\n                archive-name     \"a local package archive\")\n        (with-temp-buffer\n          (insert-file-contents (expand-file-name \"eldev.el\" eldev-local))\n          (setf eldev-pkg                    (package-buffer-info)\n                (package-desc-dir eldev-pkg) (expand-file-name eldev-local))\n          ;; Currently Eldev has no external dependencies, but let's be generic.\n          (dolist (requirement (package-desc-reqs eldev-pkg))\n            (unless (package-activate (car requirement))\n              (push requirement requirements))))))\n    (when (if eldev-pkg\n              requirements\n            (not (package-activate 'eldev)))\n      (let ((inhibit-message nil))\n        (message \"Bootstrapping Eldev for Emacs %s from %s...\\n\" eldev--emacs-version archive-name)\n        (when eldev-pkg\n          (message \"Eldev package itself will be used from `%s'\\n\" eldev-local)))\n      ;; See `eldev-retrying-for-robustness'; since Eldev is not bootstrapped yet, we have\n      ;; to inline everything.  No control from command line here.\n      (let* ((all-retry-delays (when (equal (getenv \"CI\") \"true\") '(30 60 120 180 300)))\n             (remaining-delays all-retry-delays))\n        (catch 'obtained-result\n          (while t\n            (condition-case error\n                (throw 'obtained-result (let ((debug-on-error (and debug-on-error (null remaining-delays))))\n                                          ;; See similar workarounds for `package-refresh-contents' in `eldev.el'.\n                                          (let* (failure\n                                                 (failure-catcher (lambda (original archive &rest arguments)\n                                                                    (unless failure\n                                                                      (condition-case-unless-debug error\n                                                                          (apply original archive arguments)\n                                                                        (error (setf failure (cons error (if (consp archive) (car archive) archive)))))))))\n                                            (advice-add 'package--download-one-archive :around failure-catcher)\n                                            (unwind-protect\n                                                (package-refresh-contents)\n                                              (advice-remove 'package--download-one-archive failure-catcher))\n                                            (when failure\n                                              (error \"%s (when updating contents of package archive `%s')\" (error-message-string (car failure)) (cdr failure))))))\n              (error (let ((inhibit-message nil)\n                           (delay           (pop remaining-delays)))\n                       (unless delay\n                         (when all-retry-delays\n                           (message \"Giving up: too many retries already\"))\n                         (signal (car error) (cdr error)))\n                       (message \"%s\" (error-message-string error))\n                       (message \"Assuming this is an intermittent problem, waiting %s before retrying...\\n\"\n                                (if (< delay 60) (format \"%s s\" delay) (format \"%s m\" (/ delay 60))))\n                       (sleep-for delay)\n                       (let ((n (- 5 (length remaining-delays))))\n                         (message \"Retry #%d%s...\" n (if (= n 5) \", the last\" \" of maximum 5\")))))))))\n      (if eldev-pkg\n          (package-download-transaction (package-compute-transaction nil requirements))\n        (package-install 'eldev)))\n    (when eldev-pkg\n      (push `(eldev . (,eldev-pkg)) package-alist)\n      ;; `package--autoloads-file-name' is package-private.\n      (let* ((autoloads-file     (expand-file-name (format \"%s-autoloads\" (package-desc-name eldev-pkg))\n                                                   (package-desc-dir eldev-pkg)))\n             (autoloads-disabler (lambda (do-load file &rest args) (unless (equal file autoloads-file) (apply do-load file args)))))\n        ;; Otherwise old Emacs versions print an ugly error having not found the autoloads file.\n        (advice-add #'load :around autoloads-disabler)\n        (package-activate-1 eldev-pkg)\n        ;; As of commit 1d5b164109b in Emacs repository, `package-activate-1' no longer modifies `load-path',\n        ;; leaving this to the autoloads file.  As we don't have such a file, we have to do that ourselves.\n        (add-to-list 'load-path (package-desc-dir eldev-pkg))\n        (advice-remove #'load autoloads-disabler))))\n  (require 'eldev)\n  (eldev-start-up))" "--execute" "(kill-emacs (eldev-cli (append (cdr (member \"--\" command-line-args)) nil)))" "--" "--use-emacsloadpath" "-dtTC" "test"))
               command-line()
               normal-top-level()
...

[doublep]

I guess this means that Eldev still needs to add the current directory to `load-path'. I presumed that I could use the value set via EMACSLOADPATH completely as-is.

To confirm, can you run e.g.:

$ eldev --use-emacsloadpath eval --dont-require load-path

and tell me what it prints out?

Paul

On Sat, 17 Feb 2024 at 17:34, suhail-singh @.***> wrote:

I was not able to get the build to work on commit 92de7f9 https://github.com/emacs-eldev/eldev/commit/92de7f9c62b202583f0efe9b46c89b04838e3cad using --use-emacsloadpath.

When using the below comandline invocation to build a package (say, pkg-foo):

eldev --use-emacsloadpath -dtTC test

The following exception is raised:

... [00:00.035] Debugger entered--Lisp error: (error "Unable to unload package ?pkg-foo?") signal(error ("Unable to unload package ?pkg-foo?")) error("Unable to unload package `%s'" pkg-foo) eldev--unload-package(pkg-foo) eldev--do-build((":autoloads")) eldev-build(":autoloads")

f(compiled-function (type additional-sets) #<bytecode 0x5a63cfd5ffc4b40>)(t (test))

           run-hook-with-args(#f(compiled-function (type additional-sets) #<bytecode 0x5a63cfd5ffc4b40>) t (test))
           eldev-load-project-dependencies(test)
           eldev--do-test((ert buttercup) nil)
           eldev-test()
           apply(eldev-test nil)
           eldev--execute-command(("test"))
           eldev-cli(("--use-emacsloadpath" "-dtTC" "test"))
           (kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil)))
           eval((kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil))) t)
           command-line-1(("--execute" "(let ((eldev--emacs-version (format \"%s.%s\" emacs-major-version emacs-minor-version))\n      (eldev--dir           (getenv \"ELDEV_DIR\"))\n      ;; This is intentional.  First, this is in case ELDEV_LOCAL is\n      ;; defined, second, this is just Eldev default for packages.\n      (load-prefer-newer    t))\n  ;; Setting `debug-on-error' would be useful, but it can break many\n  ;; `package-*' functions, since those use `with-demoted-errors' and\n  ;; so `condition-case-unless-debug'.\n  (unless (and (fboundp 'version<=) (version<= \"24.1\" eldev--emacs-version))\n    (error \"Eldev requires Emacs 24.1 or newer\"))\n  (setf package-user-dir\n        (expand-file-name \"bootstrap\"\n                          (expand-file-name eldev--emacs-version\n                                            (if (> (length eldev--dir) 0)\n                                                eldev--dir\n                                              (if (file-directory-p \"~/.eldev\")\n                                                  \"~/.eldev\"\n                                                ;; Duplicating not-yet-available code from `eldev-xdg-cache-home'.\n                                                (expand-file-name \"eldev\"\n                                                                  (let ((eldev--xdg-cache-dir (getenv \"XDG_CACHE_HOME\")))\n                                                                    (if (and eldev--xdg-cache-dir (file-name-absolute-p eldev--xdg-cache-dir))\n                                                                        eldev--xdg-cache-dir\n                                                                      \"~/.cache\")))))))\n        package-directory-list nil\n        package-archives       nil)\n  (require 'package)\n  (package-initialize t)\n  (let ((package-archives '((\"melpa-stable\" . \"http://stable.melpa.org/packages/\")))\n        (archive-name      \"MELPA Stable\")\n        (inhibit-message  t)\n        (eldev-local      (getenv \"ELDEV_LOCAL\"))\n        eldev-pkg\n        requirements)\n    (unless (= (length eldev-local) 0)\n      (if (string-prefix-p \":pa:\" eldev-local)\n          (setf package-archives `((\"bootstrap-pa\" . ,(file-name-as-directory (substring eldev-local (length \":pa:\")))))\n                archive-name     \"a local package archive\")\n        (with-temp-buffer\n          (insert-file-contents (expand-file-name \"eldev.el\" eldev-local))\n          (setf eldev-pkg                    (package-buffer-info)\n                (package-desc-dir eldev-pkg) (expand-file-name eldev-local))\n          ;; Currently Eldev has no external dependencies, but let's be generic.\n          (dolist (requirement (package-desc-reqs eldev-pkg))\n            (unless (package-activate (car requirement))\n              (push requirement requirements))))))\n    (when (if eldev-pkg\n              requirements\n            (not (package-activate 'eldev)))\n      (let ((inhibit-message nil))\n        (message \"Bootstrapping Eldev for Emacs %s from %s...\\n\" eldev--emacs-version archive-name)\n        (when eldev-pkg\n          (message \"Eldev package itself will be used from `%s'\\n\" eldev-local)))\n      ;; See `eldev-retrying-for-robustness'; since Eldev is not bootstrapped yet, we have\n      ;; to inline everything.  No control from command line here.\n      (let* ((all-retry-delays (when (equal (getenv \"CI\") \"true\") '(30 60 120 180 300)))\n             (remaining-delays all-retry-delays))\n        (catch 'obtained-result\n          (while t\n            (condition-case error\n                (throw 'obtained-result (let ((debug-on-error (and debug-on-error (null remaining-delays))))\n                                          ;; See similar workarounds for `package-refresh-contents' in `eldev.el'.\n                                          (let* (failure\n                                                 (failure-catcher (lambda (original archive &rest arguments)\n                                                                    (unless failure\n                                                                      (condition-case-unless-debug error\n                                                                          (apply original archive arguments)\n                                                                        (error (setf failure (cons error (if (consp archive) (car archive) archive)))))))))\n                                            (advice-add 'package--download-one-archive :around failure-catcher)\n                                            (unwind-protect\n                                                (package-refresh-contents)\n                                              (advice-remove 'package--download-one-archive failure-catcher))\n                                            (when failure\n                                              (error \"%s (when updating contents of package archive `%s')\" (error-message-string (car failure)) (cdr failure))))))\n              (error (let ((inhibit-message nil)\n                           (delay           (pop remaining-delays)))\n                       (unless delay\n                         (when all-retry-delays\n                           (message \"Giving up: too many retries already\"))\n                         (signal (car error) (cdr error)))\n                       (message \"%s\" (error-message-string error))\n                       (message \"Assuming this is an intermittent problem, waiting %s before retrying...\\n\"\n                                (if (< delay 60) (format \"%s s\" delay) (format \"%s m\" (/ delay 60))))\n                       (sleep-for delay)\n                       (let ((n (- 5 (length remaining-delays))))\n                         (message \"Retry #%d%s...\" n (if (= n 5) \", the last\" \" of maximum 5\")))))))))\n      (if eldev-pkg\n          (package-download-transaction (package-compute-transaction nil requirements))\n        (package-install 'eldev)))\n    (when eldev-pkg\n      (push `(eldev . (,eldev-pkg)) package-alist)\n      ;; `package--autoloads-file-name' is package-private.\n      (let* ((autoloads-file     (expand-file-name (format \"%s-autoloads\" (package-desc-name eldev-pkg))\n                                                   (package-desc-dir eldev-pkg)))\n             (autoloads-disabler (lambda (do-load file &rest args) (unless (equal file autoloads-file) (apply do-load file args)))))\n        ;; Otherwise old Emacs versions print an ugly error having not found the autoloads file.\n        (advice-add #'load :around autoloads-disabler)\n        (package-activate-1 eldev-pkg)\n        ;; As of commit 1d5b164109b in Emacs repository, `package-activate-1' no longer modifies `load-path',\n        ;; leaving this to the autoloads file.  As we don't have such a file, we have to do that ourselves.\n        (add-to-list 'load-path (package-desc-dir eldev-pkg))\n        (advice-remove #'load autoloads-disabler))))\n  (require 'eldev)\n  (eldev-start-up))" "--execute" "(kill-emacs (eldev-cli (append (cdr (member \"--\" command-line-args)) nil)))" "--" "--use-emacsloadpath" "-dtTC" "test"))
           command-line()
           normal-top-level()

...

— Reply to this email directly, view it on GitHub https://github.com/emacs-eldev/eldev/issues/99#issuecomment-1950250969, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYPBIJOVEO7OR4TSZIO43YUDLY7AVCNFSM6AAAAABCLE5D7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGI2TAOJWHE . You are receiving this because you commented.Message ID: @.***>

[suhail-singh]

I guess this means that Eldev still needs to add the current directory to`load-path'.

I don't believe so. But I do believe that some pre-condition you're assuming is too strict. Specifically, the load-path does contain a directory that contains the checkout of the package that is being built. Note, however, this directory may not always be the current working directory (I am not sure if this is relevant to what you're doing, but noting here for clarity).

To make the below concrete, the package in question I'm testing is called emacs-ox-tufte in Guix.

eldev --use-emacsloadpath eval --dont-require load-path

This doesn't work, and fails with:

Unable to unload package ‘ox-tufte’
Run with ‘--debug’ (‘-d’) option to see error backtrace

Instead, I tried the below:

eldev -S '(error load-path)'

Which results in (I've inserted newlines below in the output to make it easier to read):

Wrong type argument: stringp,
("/tmp/guix-build-emacs-ox-tufte-4.0.4.drv-0/source"
"/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/site-lisp"
"/gnu/store/4wmibg2k9j7jxswqfzcd729q4bdq6wd1-emacs-buttercup-1.33/share/emacs/site-lisp/buttercup-1.33"
"/gnu/store/9rnd4kygmadvwhz6sl60w8dyjw9qr0bf-emacs-eldev-1.8.3snapshot/share/emacs/site-lisp/eldev-1.8.3snapshot"
"/gnu/store/kgvi1dv7kkr47yxn66jrpdsgzp807af5-emacs-org-9.6.17/share/emacs/site-lisp/org-9.6.17" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/vc" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/use-package" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/url" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/textmodes" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/progmodes" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/play" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/org" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/nxml" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/net" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/mh-e" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/mail" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/leim" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/language" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/international" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/image" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/gnus" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/eshell" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/erc" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/emulation" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/emacs-lisp" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/cedet" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/calendar" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/calc" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/obsolete" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/vc" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/use-package" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/url" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/textmodes" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/progmodes" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/play" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/org" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/nxml" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/net" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/mh-e" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/mail" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/leim" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/language" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/international" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/image" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/gnus" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/eshell" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/erc" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/emulation" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/emacs-lisp" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/cedet" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/calendar" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/calc" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp/obsolete" "/gnu/store/3ki76bxalfm5vpkk9jbddnw4gk420svv-emacs-minimal-29.1/share/emacs/29.1/lisp")
Failed setup step: evaluating form ‘(error load-path)’ specified on the command line
Run with ‘--debug’ (‘-d’) option to see error backtrace

The first entry in the load-path ("/tmp/guix-build-emacs-ox-tufte-4.0.4.drv-0/source") contains a checkout of the package being built. Since this was my n-th rebuild my current directory is /tmp/guix-build-emacs-ox-tufte-4.0.4.drv-n/source, which is distinct from the above, not that that should really matter.

It seems the issue may be with eldev--package-load-paths. Specifically, it is nil.

eldev -S '(error eldev--package-load-paths)'

Results in:

Wrong type argument: stringp, nil
Failed setup step: evaluating form ‘(error eldev--package-load-paths)’ specified on the command line
Run with ‘--debug’ (‘-d’) option to see error backtrace

[doublep]

Ah, you are right, I didn't pay enough attention to the error message. It seems like it's caused by `eldev--unload-package', and I need to remember first what it's for.

Speaking of which, is there a way I could test this myself? E.g. some Docker image or something? Would be much easier than going back and forth with "please try now".

Paul

On Sat, 17 Feb 2024 at 17:34, suhail-singh @.***> wrote:

I was not able to get the build to work on commit 92de7f9 https://github.com/emacs-eldev/eldev/commit/92de7f9c62b202583f0efe9b46c89b04838e3cad using --use-emacsloadpath.

When using the below comandline invocation to build a package (say, pkg-foo):

eldev --use-emacsloadpath -dtTC test

The following exception is raised:

... [00:00.035] Debugger entered--Lisp error: (error "Unable to unload package ?pkg-foo?") signal(error ("Unable to unload package ?pkg-foo?")) error("Unable to unload package `%s'" pkg-foo) eldev--unload-package(pkg-foo) eldev--do-build((":autoloads")) eldev-build(":autoloads")

f(compiled-function (type additional-sets) #<bytecode 0x5a63cfd5ffc4b40>)(t (test))

           run-hook-with-args(#f(compiled-function (type additional-sets) #<bytecode 0x5a63cfd5ffc4b40>) t (test))
           eldev-load-project-dependencies(test)
           eldev--do-test((ert buttercup) nil)
           eldev-test()
           apply(eldev-test nil)
           eldev--execute-command(("test"))
           eldev-cli(("--use-emacsloadpath" "-dtTC" "test"))
           (kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil)))
           eval((kill-emacs (eldev-cli (append (cdr (member "--" command-line-args)) nil))) t)
           command-line-1(("--execute" "(let ((eldev--emacs-version (format \"%s.%s\" emacs-major-version emacs-minor-version))\n      (eldev--dir           (getenv \"ELDEV_DIR\"))\n      ;; This is intentional.  First, this is in case ELDEV_LOCAL is\n      ;; defined, second, this is just Eldev default for packages.\n      (load-prefer-newer    t))\n  ;; Setting `debug-on-error' would be useful, but it can break many\n  ;; `package-*' functions, since those use `with-demoted-errors' and\n  ;; so `condition-case-unless-debug'.\n  (unless (and (fboundp 'version<=) (version<= \"24.1\" eldev--emacs-version))\n    (error \"Eldev requires Emacs 24.1 or newer\"))\n  (setf package-user-dir\n        (expand-file-name \"bootstrap\"\n                          (expand-file-name eldev--emacs-version\n                                            (if (> (length eldev--dir) 0)\n                                                eldev--dir\n                                              (if (file-directory-p \"~/.eldev\")\n                                                  \"~/.eldev\"\n                                                ;; Duplicating not-yet-available code from `eldev-xdg-cache-home'.\n                                                (expand-file-name \"eldev\"\n                                                                  (let ((eldev--xdg-cache-dir (getenv \"XDG_CACHE_HOME\")))\n                                                                    (if (and eldev--xdg-cache-dir (file-name-absolute-p eldev--xdg-cache-dir))\n                                                                        eldev--xdg-cache-dir\n                                                                      \"~/.cache\")))))))\n        package-directory-list nil\n        package-archives       nil)\n  (require 'package)\n  (package-initialize t)\n  (let ((package-archives '((\"melpa-stable\" . \"http://stable.melpa.org/packages/\")))\n        (archive-name      \"MELPA Stable\")\n        (inhibit-message  t)\n        (eldev-local      (getenv \"ELDEV_LOCAL\"))\n        eldev-pkg\n        requirements)\n    (unless (= (length eldev-local) 0)\n      (if (string-prefix-p \":pa:\" eldev-local)\n          (setf package-archives `((\"bootstrap-pa\" . ,(file-name-as-directory (substring eldev-local (length \":pa:\")))))\n                archive-name     \"a local package archive\")\n        (with-temp-buffer\n          (insert-file-contents (expand-file-name \"eldev.el\" eldev-local))\n          (setf eldev-pkg                    (package-buffer-info)\n                (package-desc-dir eldev-pkg) (expand-file-name eldev-local))\n          ;; Currently Eldev has no external dependencies, but let's be generic.\n          (dolist (requirement (package-desc-reqs eldev-pkg))\n            (unless (package-activate (car requirement))\n              (push requirement requirements))))))\n    (when (if eldev-pkg\n              requirements\n            (not (package-activate 'eldev)))\n      (let ((inhibit-message nil))\n        (message \"Bootstrapping Eldev for Emacs %s from %s...\\n\" eldev--emacs-version archive-name)\n        (when eldev-pkg\n          (message \"Eldev package itself will be used from `%s'\\n\" eldev-local)))\n      ;; See `eldev-retrying-for-robustness'; since Eldev is not bootstrapped yet, we have\n      ;; to inline everything.  No control from command line here.\n      (let* ((all-retry-delays (when (equal (getenv \"CI\") \"true\") '(30 60 120 180 300)))\n             (remaining-delays all-retry-delays))\n        (catch 'obtained-result\n          (while t\n            (condition-case error\n                (throw 'obtained-result (let ((debug-on-error (and debug-on-error (null remaining-delays))))\n                                          ;; See similar workarounds for `package-refresh-contents' in `eldev.el'.\n                                          (let* (failure\n                                                 (failure-catcher (lambda (original archive &rest arguments)\n                                                                    (unless failure\n                                                                      (condition-case-unless-debug error\n                                                                          (apply original archive arguments)\n                                                                        (error (setf failure (cons error (if (consp archive) (car archive) archive)))))))))\n                                            (advice-add 'package--download-one-archive :around failure-catcher)\n                                            (unwind-protect\n                                                (package-refresh-contents)\n                                              (advice-remove 'package--download-one-archive failure-catcher))\n                                            (when failure\n                                              (error \"%s (when updating contents of package archive `%s')\" (error-message-string (car failure)) (cdr failure))))))\n              (error (let ((inhibit-message nil)\n                           (delay           (pop remaining-delays)))\n                       (unless delay\n                         (when all-retry-delays\n                           (message \"Giving up: too many retries already\"))\n                         (signal (car error) (cdr error)))\n                       (message \"%s\" (error-message-string error))\n                       (message \"Assuming this is an intermittent problem, waiting %s before retrying...\\n\"\n                                (if (< delay 60) (format \"%s s\" delay) (format \"%s m\" (/ delay 60))))\n                       (sleep-for delay)\n                       (let ((n (- 5 (length remaining-delays))))\n                         (message \"Retry #%d%s...\" n (if (= n 5) \", the last\" \" of maximum 5\")))))))))\n      (if eldev-pkg\n          (package-download-transaction (package-compute-transaction nil requirements))\n        (package-install 'eldev)))\n    (when eldev-pkg\n      (push `(eldev . (,eldev-pkg)) package-alist)\n      ;; `package--autoloads-file-name' is package-private.\n      (let* ((autoloads-file     (expand-file-name (format \"%s-autoloads\" (package-desc-name eldev-pkg))\n                                                   (package-desc-dir eldev-pkg)))\n             (autoloads-disabler (lambda (do-load file &rest args) (unless (equal file autoloads-file) (apply do-load file args)))))\n        ;; Otherwise old Emacs versions print an ugly error having not found the autoloads file.\n        (advice-add #'load :around autoloads-disabler)\n        (package-activate-1 eldev-pkg)\n        ;; As of commit 1d5b164109b in Emacs repository, `package-activate-1' no longer modifies `load-path',\n        ;; leaving this to the autoloads file.  As we don't have such a file, we have to do that ourselves.\n        (add-to-list 'load-path (package-desc-dir eldev-pkg))\n        (advice-remove #'load autoloads-disabler))))\n  (require 'eldev)\n  (eldev-start-up))" "--execute" "(kill-emacs (eldev-cli (append (cdr (member \"--\" command-line-args)) nil)))" "--" "--use-emacsloadpath" "-dtTC" "test"))
           command-line()
           normal-top-level()

...

— Reply to this email directly, view it on GitHub https://github.com/emacs-eldev/eldev/issues/99#issuecomment-1950250969, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYPBIJOVEO7OR4TSZIO43YUDLY7AVCNFSM6AAAAABCLE5D7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGI2TAOJWHE . You are receiving this because you commented.Message ID: @.***>

[suhail-singh]

Speaking of which, is there a way I could test this myself? E.g. some Docker image or something?

Depends on your appetite for yak-shaving. Since I'm building in Guix, doing builds and possibly rebuilds (when debugging) would require some (transfer of) Guix knowledge.

Would be much easier than going back and forth with "please try now".

Another alternative may be having a paired programming session.

[doublep]

I looked close to it now. The problem is with building and requiring autoloads, in the project itself and its dependencies. Since this heavily depends on the normal dependency-as-package management, it gets broken if those are disabled. The easiest workaround, of course, is to completely turn autoload handling off when using `--disable-dependencies' and just document it like that. After all, this new option is only supposed to be "for special needs", not something recommended for normal use.

Would it be fine for you, or do you need autoloads, e.g. during testing, when running on Guix too?

Paul

On Sat, 17 Feb 2024 at 21:44, suhail-singh @.***> wrote:

Speaking of which, is there a way I could test this myself? E.g. some Docker image or something?

Depends on your appetite for yak-shaving. Since I'm building in Guix, doing builds and possibly rebuilds (when debugging) would require some (transfer of) Guix knowledge.

Would be much easier than going back and forth with "please try now".

Another alternative may be having a paired programming session. My email is suhail at-sign bayesians dot ca, in case you'd like to coordinate a joint call to debug/resolve this.

— Reply to this email directly, view it on GitHub https://github.com/emacs-eldev/eldev/issues/99#issuecomment-1950309811, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYPBPH5GOEZXWQUOAL32TYUEJDXAVCNFSM6AAAAABCLE5D7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJQGMYDSOBRGE . You are receiving this because you commented.Message ID: @.***>

[suhail-singh]

Would it be fine for you, or do you need autoloads, e.g. during testing, when running on Guix too?

Could you explain via a concrete example what "needing autoloads" would look like when testing a package (say, pkg-foo)?

Would it be that autoloads for package pkg-foo wouldn't get generated? Something else?

[doublep]

They wouldn't be generated and wouldn't take any effect, unlike with normal Eldev operation. You would need to explicitly require the feature declaring a function before calling it, even if the function was declared as `###autoload'.

Paul

On Sun, 18 Feb 2024 at 16:15, suhail-singh @.***> wrote:

Would it be fine for you, or do you need autoloads, e.g. during testing, when running on Guix too?

Could you explain via a concrete example what "needing autoloads" would look like when testing a package (say, pkg-foo)?

Would it be that autoloads for package pkg-foo wouldn't get generated? Something else?

— Reply to this email directly, view it on GitHub https://github.com/emacs-eldev/eldev/issues/99#issuecomment-1951354997, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYPBJW57UV27UX5GI7NGLYUILIJAVCNFSM6AAAAABCLE5D7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJRGM2TIOJZG4 . You are receiving this because you commented.Message ID: @.***>

[suhail-singh]

I believe it should be okay; let's try it and see?

[doublep]

Please test after commit be713dd.

Paul

On Mon, 19 Feb 2024 at 00:44, suhail-singh @.***> wrote:

I believe it should be okay; let's try it and see?

— Reply to this email directly, view it on GitHub https://github.com/emacs-eldev/eldev/issues/99#issuecomment-1951486894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABYPBKEP5Z4CC5ZLZOL6ODYUKG6VAVCNFSM6AAAAABCLE5D7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJRGQ4DMOBZGQ . You are receiving this because you commented.Message ID: @.***>

[suhail-singh]

Please test after commit be713dd.

Yeah, that works well. On Guix, it's sufficient for the purposes of running tests for a package (as part of the build process) that uses Eldev.

The previous invocation can now be simplified to:

eldev --use-emacsloadpath -dtTC test

Thank you!

[suhail-singh]

Out of curiosity, are there any estimates for when version 1.9 will be released?

[doublep]

Sorry, I was busy with unrelated things and then forgot about this issue. Before releasing 1.9 I want to figure out the problems in #100, but there are no other plans.

[doublep]

Released 1.9 with the new option.