Reintroduce Makefile wrapping eldev

[arichiardi]

Hi Bohzidar and team cider!

This might be a long shot but I would like to propose the re-introduction of Makefile in this repo to wrap the eldev compilation.

I am using borg to manage my configuration and borg works really well if a package compiles using make...aka I don't basically need to maintain a script that I can call when, say, I recompile my emacs.

That change was quite breaking for my workflow and because of time I decided to leave the cider sources uncompiled until I could find time to propose an alternative solution to the problem.

I can of course do the work, which is quite straight-forward...thoughts?

[bbatsov]

That's fine by me, if it's causing issues for borg users, although I wonder how common are makefiles within Emacs packages in general. Just put some note in the Makefile it's there for borg compatibility and people are encouraged to use eldev directly.

@doublep I guess you can look into some eldev <-> borg integration down the road.

[doublep]

I would say that borg is the higher level here. It needs to adapt, not Eldev or projects that use it. Would also be much more generic.

[arichiardi]

Ok both of your messages made me think maybe borg needs to get eldev support, let me ask @tarsius opinion and if that's the case this issue can be closed.

Thanks!

[tarsius]

99% of all packages are trivial to compile, so borg assumes each package is trivial until told otherwise. borg compiles trivial package about the same way as package compiles them.

Some packages are more complex, so borg can be told how to compile those by specifying build steps explicitly. Here are two examples:

[submodule "libgit"]
    ...
    build-step = make

[submodule "notmuch"]
    ...
    load-path = emacs
    build-step = ./configure
    build-step = make
    build-step = borg-update-autoloads

I assume that @arichiardi previously (until 3d8552a8469a5515cd6e783b5a9ab09ba9d51eb5) used something like:

[submodule "cider"]
    ...
    build-step = make

He could have used this instead:

[submodule "cider"]
    ...
    build-step = cask install
    build-step = cask update
    build-step = cask build

Which is about what make does, except that that avoids the install and update subcommands if those are unnecessary. This also fails to generate the autoloads file, though maybe Cask provides a command for that as well, I don't know. If not, appending this should work:

    build-step = borg-update-autoloads

Now that Cask has been replaced by Eldev and the Makefile is gone, the new appropriate build-steps are:

[submodule "cider"]
    ...
    build-step = eldev build :autoloads

Or so I assume after skimming the Eldev documentation and Cider's "doc/modules/ROOT/pages/contributing/hacking.adoc".

Eldev and Borg take fundamentally different approaches when it comes to separating the everyday Emacs installation from the build environment used during development. In the case of Eldev it appears to be a major goal to provide that separation, while Borg's approach is to acknowledge that trying to do so is futile.

Because of that I advice Borg users to not wrap around Cask or Eldev. Instead they should just rely on Borg's trivial compilation mechanism whenever possible. Try it!

I think that should work in case of Cider. Just because a package comes with Make-/Cask-/Eldav- file does not mean that it is actually necessary to use that. Magit for example comes with a Makefile that tries to deal many different situations, but I don't rely on that as a user of Magit who installed is using Borg. When I work on it, then I often run make, but I could also just use Borg (cd ../..; make lib/magit).

---

Cask/Eldev don't use the installed versions of a package's dependencies and that seems to be a major selling point. Provided you make absolutely sure that these two environment actually stay separate, that seems okay, but if you end up compiling a package against one version of a dependency and then use a different version of that dependency at runtime, then that can lead to very strange bugs that cannot be reproduced on other machines.

For example, if foo defines a macro foo-loop that expands to the foo-loop--internal function and you compile your package against that, then things will get confusing if you run it against a new version of foo that removes foo-loop--internal and changes foo-loop to not contain that function in the expansion. You will get a backtrace that references foo-loop--internal but you won't find any source code that actually contains foo-loop--internal and that... can be confusing.

So Borg users should never specify build steps that use Cask or Eldev or anything like that.

---

IMO it is a mistake for Cask/Eldev to even be using separate builds of the dependencies. Emacs Lisp ('s runtime environment) is not like, for example, Javascript or Rust. Emacs packages are not run as independent web services or compiled into independent binaries. Instead all packages are loaded into the same Emacs Lisp (virtual) machine. There are also no modules/packages (in the sense used by languages other than Emacs Lips). You cannot load different versions of a library for different packages to use in the same Emacs instance.

You know all this of course. Where we differ is that I think that this means that using different build artifacts of dependencies while developing does not actually solve any problems. As long as you test you package against only one version of a given dependency, it might as well be the same version as the one you are using in your daily (non-developer) Emacs usage.

[arichiardi]

So Borg users should never specify build steps that use Cask or Eldev or anything like that.

Thank you Jonas for chiming in, I am actually always learning something when you or Bozhidar speak :smile:

The command I am actually running out of band is eldev compile && eldev build :autoloads.

Instead they should just rely on Borg's trivial compilation mechanism whenever possible. Try it!

And this is what I am going to try before using build-steps.

Closing this one now, the solution being very clear and the answer thorough.

:bow:

[doublep]

@arichiardi: Since 0.8 Eldev provides option -X to use preinstalled packages, e.g. in Emacs directory ~/.emacs.d/elpa.

[tarsius]

Nice!