macOS; emacs 28.2 built with --with-ns; make bootstrap hits: Symbol's function definition is void: debug

[pedz]

I am trying to carefully step through Bootstrapping from scratch. I am on macOS using emacs 28.2. I build this emacs myself via configure --with-ns (which is key). I have a symbolic link from emacs to the Emacs executable inside the Emacs.app which "works" as far as I have experienced thus far.

I got the follow error:

make bootstrap

=== Running 'git submodule init' ===

=== Running 'lib/borg/borg.sh' ===
--- [borg] ---

=== Running 'make build' ===

emacs -Q --batch  --load subr-x --eval "(setq byte-compile-warnings '(not docstrings))" --eval "(fset 'original-message (symbol-function 'message))" --eval "(fset 'message (lambda (format &rest args) (unless (or (equal format \"pcase-memoize: equal first branch, yet different\") (equal format \"Not registering prefix \\\"%s\\\" from %s.  Affects: %S\") (and (stringp (car args)) (string-match-p \"Scraping files for\" (car args)))) (apply 'original-message format args))))" \
    -L lib/borg/ --load borg --funcall borg-initialize \
    --funcall borg-batch-rebuild  2>&1
Warning: arch-dependent data dir 'Contents/MacOS/libexec/': No such file or directory
Warning: Lisp directory 'Contents/Resources/lisp': No such file or directory
Symbol's function definition is void: debug
make[1]: *** [build] Error 255
make: *** [bootstrap] Error 2

I then re-extracted fresh from the 28.2 tarball, did just ./configure && make and then did make EMACS=/Users/pedz/Source/emacs-28.2/src/emacs bootstrap which worked.

I doubt very much if this has anything to do with your repo but I thought I would report it since I found a work around in case others hit the same issue.

I've not been able (yet) to step through the process of recreating the error with debug-on-error set but I really want to see if I can. This appears to be a bug with the distribution which I'd like to find and point out to the maintainers.

I'll welcome any suggestions on how to track down the bug.

[pedz]

Update:

I don't understand this but using the original --with-ns build, make EMACS=/Applications/Emacs.app/Contents/MacOS/Emacs bootstrap works but make bootstrap does not work where I have a symlink:

lrwxr-xr-x  1 pedz  staff  44 Apr 13  2022 /Users/pedz/bin/emacs -> /Applications/Emacs.app/Contents/MacOS/Emacs

[pedz]

Another update (perhaps should be a separate issue). The instructions on Bootstrapping from scratch say make all in the set of commands after the first commit. But there is no all target. In Make Targets you suggest to create an all target so perhaps to simplify things, that line should be make clean build or change your suggestion for the initial Makefile.

[tarsius]

I've changed it to make build.

[tarsius]

I'll welcome any suggestions on how to track down the bug.

I can't think of anything. So... debug statements, all over the place.

[pedz]

I hope I don't over stay my welcome 😀. Two items:

1) I did epkg-describe-package and I got a prompt to Clone epkgs. I didn't notice the 's' so I said No assuming I had screwed something up but eventually figured out it was the DB so I said "yes". So perhaps change "Clone" to something like "Clone the epkg database ..." I can do a pull request if you want me to.

2) You have a call to file-name-split in epkg.el. I found it in the master branch of emacs but (according to my rather weak git abilities) it is not in the emacs-28 branch. I'll probably make a local copy of the function and carry on.

[tarsius]

1. This prompt?: Clone https://github.com/emacsmirror/epkgs.git to ~/.config/emacs/epkgs/? (y or n). That's already fairly long, so I wouldn't want to add anything to it, but I suppose I could replace the url with the package database, which is less precise, but I could additionally add repository. Not sure that would be an improvement.
2. I've added epkg--file-name-split.

[pedz]

Yes. That prompt. The other idea I had is just mention that the database is going to get cloned in your manual. Which leads to another question I had but figured I would ask later... how / when does it get updated? Do I update that repo by hand?

[pedz]

Well... right now, I don't think the symlink problem can be fixed. Launching Emacs via a symlink fails while launching it via an absolute path works. Here's why:

Starting from scratch, the original error message told us what was wrong. Emacs can not find the lisp directory which starts out as a relative path of Contents/Resources/lisp. Eventually Emacs tries to call normal-top-level in startup.el which is the loop that parses the command line, etc. But since Emacs can't find the lisp directory, it can't find startup.el so it throws an error. That tries to take us to debug which is in debug.el but Emacs can't find that either so Emacs just gives up.

But what is the difference between starting from a symbolic link verses starting from an absolute path?

Emacs asks macOS "what is my mainBundle?" and then, from that, asks "what is my mainBundle's bundlePath?" which is the absolute path to the application's directory. (Applications on macOS are essentially just specially structured directories.)

In the case of launching Emacs via an absolute path, bundlePath returns sensibly with the path to the application and the lisp directory is a relative path within that directory so Emacs can find the directory and is happy.

In the case of launching via a symbolic link, macOS returns the path to the directory containing the symbolic link as its bundlePath -- which isn't correct. Emacs can't find the lisp directory because it is starting from the wrong point, and so fails.

[tarsius]

I recently had a marginally related issue myself (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58073). I dealt with that by using a wrapper script instead of a symlink. That might work for you too.

[pedz]

That was an interesting read... Creates lots of questions like How would emacs work if it can't find its dump file?

A wrapper script sounds best.

[tarsius]

Based on that commit, I assume this works now.