[bootstrap] failing to build

[rprimus]

Sun Jul 5 10:30:09 BST 2020

Hi,

System info:

---

### System Info: - **OS**: - ProductName: Mac OS X - ProductVersion: 10.15.6 - BuildVersion: 19G60d - **Emacs**: "GNU Emacs 26.3 (build 1, x86_64-apple-darwin19.6.0, Carbon Version 162 AppKit 1894.6) of 2020-06-15"

---

I've created directory /tmp/micro-borg/ initially.

I've been following the manual instructions (for using borg from scratch). I've gotten up to make bootstrap (https://github.com/emacscollective/borg/blob/master/borg.org#bootstrapping-from-scratch), but it fails. I've tracked it down to function borg-batch-rebuild:

: ; emacs -Q --batch -L lib/borg/ --load borg  --funcall borg-batch-rebuild
Args out of range: "160000 ecb2f9f0982028bd56297327621aca105ba7b28a 0   lib/borg", 50, 90

I did not run make bootstrap-borg after creating the Makefile.

Update:

Failing on borg-drones:

: ; emacs -Q --batch -L lib/borg/ --load borg  --funcall borg-drones
Args out of range: "160000 ecb2f9f0982028bd56297327621aca105ba7b28a 0   lib/borg", 50, 90

Debugger entered--Lisp error: (args-out-of-range "160000 ecb2f9f0982028bd56297327621aca105ba7b28a 0\011lib/borg" 50 90)
  substring("160000 ecb2f9f0982028bd56297327621aca105ba7b28a 0\011lib/borg" 50 90)
  (string-equal (substring line 50 offset) prefix)
  (and (string-equal (substring line 50 offset) prefix) (list (substring line offset)))
  (closure ((offset . 90) (prefix . "../../../../tmp/micro-borg/.emacs.d/lib/") (include-variables) compilation-mode-font-lock-keywords git-commit-mode-map t) (line) (and (string-equal (substring line 50 offset) prefix) (list (substring $
  mapcan((closure ((offset . 90) (prefix . "../../../../tmp/micro-borg/.emacs.d/lib/") (include-variables) compilation-mode-font-lock-keywords git-commit-mode-map t) (line) (and (string-equal (substring line 50 offset) prefix) (list (sub$
  cl-mapcan((closure ((offset . 90) (prefix . "../../../../tmp/micro-borg/.emacs.d/lib/") (include-variables) compilation-mode-font-lock-keywords git-commit-mode-map t) (line) (and (string-equal (substring line 50 offset) prefix) (list ($
  (let* ((default-directory borg-top-level-directory) (prefix (file-relative-name borg-drones-directory)) (offset (+ (length prefix) 50))) (cl-mapcan (function (lambda (line) (and (string-equal (substring line 50 offset) prefix) (list (s$
  (if include-variables (let (alist) (let ((--dolist-tail-- (and (file-exists-p borg-gitmodules-file) (process-lines "git" "config" "--list" "--file" borg-gitmodules-file)))) (while --dolist-tail-- (let ((line (car --dolist-tail--))) (if$
  borg-drones()
  eval((borg-drones) nil)
  eval-expression((borg-drones) nil nil 127)
  funcall-interactively(eval-expression (borg-drones) nil nil 127)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)

supplemental info

: ; tree
.
├── Makefile
└── lib
    └── borg
        ├── CHANGELOG
        ├── LICENSE
        ├── Makefile
        ├── README.md
        ├── borg-elpa.el
        ├── borg.el
        ├── borg.mk
        ├── borg.org
        ├── borg.sh
        └── borg.texi

: ; cat .gitmodules
[submodule "borg"]
    path = lib/borg
    url = git@github.com:emacscollective/borg.git

: ; git reflog show
ecb2f9f (HEAD -> master, origin/master, origin/HEAD) HEAD@{0}: reset: moving to ecb2f9f0982028bd56297327621aca105ba7b28a
ecb2f9f (HEAD -> master, origin/master, origin/HEAD) HEAD@{1}: clone: from git@github.com:emacscollective/borg.git

Any ideas?

[tarsius]

Sounds like something is going wrong at the very end of borg-drones. Try adding some debug statements there. Create a stash for them as they will get discarded when make bootstrap gets around to dealing with borg itself.

[tarsius]

Print the values of default-directory, borg-top-level-directory and borg-drones-directory.

[rprimus]

Sun Jul 5 11:49:27 BST 2020

Didn't realise you'd responded so quickly - before adding additional info to the original post.

: ; emacs -Q  --batch -L lib/borg/ --load borg --eval '(message "dd: %s, borg-tld: %s, borg-dd: %s" default-directory borg-top-level-directory borg-drones-directory)'
dd: /tmp/micro-borg/.emacs.d/, borg-tld: /private/tmp/micro-borg/.emacs.d/, borg-dd: /tmp/micro-borg/.emacs.d/lib/

Editing borg-drones (message debug expressions added before https://github.com/emacscollective/borg/blob/ecb2f9f0982028bd56297327621aca105ba7b28a/borg.el#L260:

: ; emacs -Q  --batch -L lib/borg/ --load borg --funcall borg-drones
*** dd: /private/tmp/micro-borg/.emacs.d/
, borg-tld: /private/tmp/micro-borg/.emacs.d/
, borg-drones-directory: /tmp/micro-borg/.emacs.d/lib/

*** prefix: ../../../../tmp/micro-borg/.emacs.d/lib/, offset: 90

Additionally:

: ; git submodule--helper list
160000 ecb2f9f0982028bd56297327621aca105ba7b28a 0   lib/borg

: ; git submodule--helper list | wc -c
      59

Looks like the error is being generated due to offset being out-of-range.

(substring "160000 ecb2f9f0982028bd56297327621aca105ba7b28a 0   lib/borg" 50 90)
*** Eval error ***  Args out of range: "160000 ecb2f9f0982028bd56297327621aca105ba7b28a 0   lib/borg", 50, 90

[tarsius]

The prefix is supposed to be lib/ but because borg-drones-directory does not begin with /private like the other directories do, the relative path is something much longer. Do you know why the others begin with /private and this does not? Is one of these just plain wrong?

[tarsius]

By the way, this is triggered by 83f96dd5bbf5f583fc2f18910c5e2eb56f0e85ba.

[rprimus]

Sun Jul 5 13:08:26 BST 2020

On macOS:

: ; ls -lad /tmp
lrwxr-xr-x 1 root admin 11 May 21 10:19 /tmp -> private/tmp

In emacs:

ELISP> default-directory
"/tmp/micro-borg/.emacs.d/"
ELISP> (process-lines "pwd")
("/private/tmp/micro-borg/.emacs.d")

[rprimus]

Sun Jul 5 13:53:43 BST 2020

Same happens under Linux:

➜  /tmp mkdir z
➜  /tmp ln -s z z-link
➜  /tmp cd z-link

ELISP> (cd "/tmp/z-link")
"/tmp/z-link/"
ELISP> default-directory
"/tmp/z-link/"
ELISP> (process-lines "pwd")
("/tmp/z")

ELISP> (call-process "/bin/pwd" nil t nil)
0 (#o0, #x0, ?\C-@)
ELISP> /tmp/z

Seems that a process returns the absolute path to the directory it is in.

[rprimus]

Sun Jul 5 14:02:12 BST 2020

From Wikipedia

Symbolic links operate transparently for many operations: programs that read or write to files named by a symbolic link will behave as if operating directly on the target file. However, they have the effect of changing an otherwise hierarchical filesystem from a tree into a directed graph, which can have consequences for such simple operations as determining the current directory of a process.

Even the Unix standard for navigating to a directory's parent directory no longer works reliably in the face of symlinks. Some shells heuristically try to uphold the illusion of a tree-shaped hierarchy, but when they do, this causes them to produce different results from other programs that manipulate pathnames without such heuristic, relying on the operating system instead.[2] Programs that need to handle symbolic links specially (e.g., shells and backup utilities) thus need to identify and manipulate them directly.

[rprimus]

Sun Jul 5 14:05:26 BST 2020

On a side-note, this would explain the problems I was having when trying borg previously. I initially had .emacs.d living under a main git repo dotfiles (not as a submodule), with ~/.emacs.d -> ~/dotfiles/.emacs.d.

borg drones would have an incorrect .git file pointing to ../../../dotfiles/.git/modules/<drone>. This was causing the error:

fatal: not a git repository (or any of the parent directories): .git

This was manually resolved by editing the .git file and removing the extra dotfiles/.

I've since moved .emacs.d into it's own git repo (using git filter-repo) and ~/.emacs.d is no longer a symlink.

(Just the added detail if someone else reports strange errors when using borg.)