[Request] Add flag to honour EMACSLOADPATH

[jellelicht]

Currently —external-deps=some-directory allows one to load external dependencies from a specific directory. In Guix, packages are built in isolation with no access to the network; ideally, we’d be able to set up dependencies for a package in :-separated string of directories via EMACSLOADPATH.

Each of the directories listed in EMACSLOADPATH is read-only, so eldev attempting to create an archive directory in the directory passed to —external-deps will consistently fail; is there any way to disable this?

[doublep]

It is not easy, because fake internal pseudoarchive --eldev-- is used to load local dependencies and, most importantly, the project being developed itself. If you simply comment out form (eldev--create-internal-pseudoarchive-descriptor), Eldev fails. Maybe this could be worked around for the project itself (local dependencies becoming disabled in such a case), hard to say.

Before doing anything here, I'd like to understand better what you are trying to achieve.

[jellelicht]

Specifically: run cider’s testsuite in the guix package definition of emacs-cider.

Running this testsuite happens in an environment without internet connectivity, but with all dependencies of cider available via EMACSLOADPATH. I believe the command is: eldev -dtT -p test.

[doublep]

As it is now, Eldev won't access internet with --external-deps, but it does need a writable directory: it creates a fake archive entry, but deletes it again after it has finished.

What would you do if Cider used some different way of testing, e.g. with Cask or using a custom Makefile? Would it be possible to instead tweak that "guix package definition" somewhat? In this case it looks like higher level compared to Eldev, and my normal reaction is that "higher level should be able to adapt".

[jellelicht]

With a custom Makefile, ideally all the write-related stuff happens in the current directory; Put another way, running make (besides make install or something) should not write things anywhere else.

Right now, it seems that eldev expects to be able to write to the directory that it is given via external-deps; why is this? Am I misunderstanding what this argument does? If eldev needs to somehow process dependencies I give it, can’t it do this in a different directory (current working directory, /tmp, some cache somewhere)?

The guix package definition can naturally be adapted to do what is needed: that is, manually find and run all the tests, just as eldev does. It would just be easier to maintain if we can leverage eldev for that 😄

[jellelicht]

And still a nitpick; it would be nice to support passing a path (: separated list of directories) via external-deps, instead of a single directory.

[doublep]

Right now, it seems that eldev expects to be able to write to the directory that it is given via external-deps; why is this? Am I misunderstanding what this argument does? If eldev needs to somehow process dependencies I give it, can’t it do this in a different directory (current working directory, /tmp, some cache somewhere)?

This directory is not simply added to load-path, it is used as the value for variable package-user-dir of Emacs packaging system. By default, in --isolated mode, subdirectory .eldev/EMACS-VERSION/packages is used, i.e. it does not match the normal Emacs you use: the project will not see packages installed in your normal Emacs, and likewise normal Emacs will not see whatever the project uses (so you can e.g. develop a project depending on Helm without ever installing Helm). However, there is --external mode in which case instead of managing dependencies itself and isolating from your normal Emacs, Eldev simply feeds all preinstalled packages to the project and hopes they will be enough. The directory used as argument to option --external is expected to be in the format used by Emacs package system, and it actually defaults to ~/.emacs.d/elpa, i.e. that of your normal Emacs.

But in either case, Eldev always operates via standard Emacs package system and not just by simply tweaking load-path. For example:

~/datetime$ eldev eval "(package-installed-p 'datetime)"
t

As you see, Emacs' package system considers the package installed, even though in reality it isn't quite so.

For this to work, Eldev creates a fake package archive (by default, this is file .eldev/EMACS-VERSION/packages/archives/--eldev--/archive-contents) that contains at least an entry for the project being developed. Additionally, it will contain an entry per local dependency if you use those. For example, I have (eldev-use-local-dependency "~/extmap" 'packaged) in ~/datetime/Eldev-local, and the file contents reflects that:

~/datetime$ cat .eldev/27.2/packages/archives/--eldev--/archive-contents
(1 (datetime . [(0 7 1) ((emacs (24 4)) (extmap (1 1 1))) nil single nil]) (extmap . [(1 2 1) ((emacs (24 1))) nil single nil]))

The problem is, this fake archive is needed (the way it is implemented) also in external dependency mode. Eldev creates it, loads the project, runs the tests (or whatever), and then deletes it as if it wasn't there, to avoid polluting this external directory. But as it is implemented, it has to be able to write there.

And still a nitpick; it would be nice to support passing a path (: separated list of directories) via external-deps, instead of a single directory.

As explained above, this is simply not possible. It is not just a part of load-path, it must be a directory with similar structure as ~/.emacs.d/elpa.

[jellelicht]

The extremely clear explanation certainly helped me finally grok it, thank you. For now I think it makes more sense to ‘simply’ run the tests via a different mechanism in the package definition of cider. If perhaps guix’s Emacs package building facilities will be based on a higher level concept (packages instead of *.el files strewn about load-path), eldev seems like a nice tool to use.

[doublep]

By the way, you might be interested in using local dependencies. As far as I can see, this is basically what you want: you have sources of the dependency packages somewhere else and want to run tests against these sources, not e.g. packages downloaded from MELPA. Then you can stay in the default "isolated" mode and the fact that it needs to write somewhere shouldn't be a problem, because this "somewhere" would be inside .eldev subdirectory. Also, you don't need to create Eldev-local for this: when wrapping Eldev with some other tool, it is often easier to use --setup (-S) options. It is heavily used in Eldev's own test set, for example.

[suhail-singh]

@jellelicht fwiw, in case you're curious, one way to invoke Eldev during Guix's check phase is to do something like what's noted in this comment.

Given the verbosity, however, it may be easier to invoke the tests directly via Emacs batch-mode.