Serious compilation errors when Spacemacs installs package realgud

[bmag]

Description

Trying to install realgud via dotspacemacs-additional-packages results in many compilation errors. The errors indicate that Emacs tries to load files from wrong paths: ~/relative/path instead of ~/.emacs.d/elpa/relative/path. Manually installing via SPC a k (spacemacs/paradox-list-packages) works without errors, only has regular compile warnings.

I've checked if the order of installing the dependencies matter, but it doesn't seem to be the problem. Stock Emacs and Spacemacs install the dependencies in a different order, but manually installing the dependecies in stock Emacs in the same order as Spacemacs uses resulted in a successful compilation.

cc @jluttine

Temporary workaround

• install realgud via SPC a k (compiles successfully)
• add realgud to dotspacemacs-additional-packages (prevents Spacemacs from deleting it as orphan)

  Reproduction guide

• add realgud to dotspacemacs-additional-packages
• make sure that list-utils, load-relative, loc-changes, test-simple and realgud aren't installed in .emacs.d/elpa
• restart spacemacs (SPC q R)

Observed behaviour: Many Cannot open load file errors during compilation. Errors indicate that Emacs tries to load files from base path ~/, instead of base path ~/.emacs.d/elpa/. Probably related to load-relative, which is a dependency of realgud. See first attached file for compile log.

Expected behaviour: Realgud is installed and compiled without errors (warning are ok, but not errors). See second attached file for compile log.

System Info

• OS: gnu/linux
• Emacs: 24.5.1
• Spacemacs: 0.105.19
• Spacemacs branch: develop (rev. 5ee970f)
• Graphic display: t
• Distribution: spacemacs
• Editing style: hybrid
• Completion: helm
• Layers:

(spacemacs-helm
 (auto-completion :variables auto-completion-complete-with-key-sequence "jk" auto-completion-complete-with-key-sequence-delay 0.5 auto-completion-enable-snippets-in-popup nil auto-completion-enable-help-tooltip t company-tooltip-align-annotations t)
 emacs-lisp git markdown org
 (shell :variables shell-default-height 30 shell-default-position 'bottom)
 syntax-checking version-control
 (latex :variables latex-enable-auto-fill t latex-enable-folding t)
 pdf-tools evil-cleverparens python command-log smex themes-megapack c-c   ruby common-lisp ibuffer octave realgud window-purpose nameless)

Compile Log when using spacemacs-addition-packages

28-100050.txt

Compile Log when using SPC a k

28-100246.txt

[d12frosted]

You could also set dotspacemacs-delete-orphan-packages to nil in order to prevent Spacemacs from deleting orphan packages.

P. S. can't help with your issue, sorry :-P

~ d12frosted

On Thu, Apr 28, 2016 at 10:20 AM, Bar notifications@github.com wrote:

Description

Trying to install realgud via dotspacemacs-additional-packages results in many compilation errors. The errors indicate that Emacs tries to load files from wrong paths: ~/relative/path instead of ~/.emacs.d/elpa/relative/path. Manually installing via SPC a k (spacemacs/paradox-list-packages) works without errors, only has regular compile warnings.

I've checked if the order of installing the dependencies matter, but it doesn't seem to be the problem. Stock Emacs and Spacemacs install the dependencies in a different order, but manually installing the dependecies in stock Emacs in the same order as Spacemacs uses resulted in a successful compilation.

cc @jluttine https://github.com/jluttine Temporary workaround

• install realgud via SPC a k (compiles successfully)
• add realgud to dotspacemacs-additional-packages (prevents Spacemacs from deleting it as orphan)

Reproduction guide

• add realgud to dotspacemacs-additional-packages
• make sure that list-utils, load-relative, loc-changes, test-simple and realgud aren't installed in .emacs.d/elpa
• restart spacemacs (SPC q R)

Observed behaviour: Many Cannot open load file errors during compilation. Errors indicate that Emacs tries to load files from base path ~/, instead of base path ~/.emacs.d/elpa/. Probably related to load-relative, which is a dependency of realgud. See first attached file for compile log.

Expected behaviour: Realgud is installed and compiled without errors (warning are ok, but not errors). See second attached file for compile log. System Info

• OS: gnu/linux
• Emacs: 24.5.1
• Spacemacs: 0.105.19
• Spacemacs branch: develop (rev. 5ee970f https://github.com/syl20bnr/spacemacs/commit/5ee970f07bcc2ae0e49554b93c4c852bd43a51d4 )
• Graphic display: t
• Distribution: spacemacs
• Editing style: hybrid
• Completion: helm
• Layers:

(spacemacs-helm (auto-completion :variables auto-completion-complete-with-key-sequence "jk" auto-completion-complete-with-key-sequence-delay 0.5 auto-completion-enable-snippets-in-popup nil auto-completion-enable-help-tooltip t company-tooltip-align-annotations t) emacs-lisp git markdown org (shell :variables shell-default-height 30 shell-default-position 'bottom) syntax-checking version-control (latex :variables latex-enable-auto-fill t latex-enable-folding t) pdf-tools evil-cleverparens python command-log smex themes-megapack c-c ruby common-lisp ibuffer octave realgud window-purpose nameless)

Compile Log when using spacemacs-addition-packages

28-100050.txt https://github.com/syl20bnr/spacemacs/files/239963/28-100050.txt Compile Log when using SPC a k

28-100246.txt https://github.com/syl20bnr/spacemacs/files/239964/28-100246.txt

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/syl20bnr/spacemacs/issues/5917

[syl20bnr]

There is something odd with load-relative-pkg.el which does not seem to detect correctly the path of the currently loaded file.

What I confirmed not to be the cause in Spacemacs install process of packages:

• using noactivate when initializing package.el seems to not change anything
• letting package.el handle the installation of dependencies leads to the same result

From there, there is not much Spacemacs does differently, Spacemacs relies on package.el via package-install function.

I guess it has to do with initialization of package.el, somehow we are missing something.

[syl20bnr]

More info.

The error seems to come from __FILE__ function in load-relative.el.

With paradox:

load-file-name: /Users/sylvain/.emacs.d/elpa/load-relative-20150224.1722/load-relative.elc
looking for: ./realgud/common/track-mode
with prefix: realgud-
;; in __FILE__
__FILE__ result is: nil

With Spacemacs:

load-file-name: /Users/sylvain/.emacs.d/elpa/load-relative-20150224.1722/load-relative.elc
looking for: ./realgud/common/track-mode
with prefix: realgud-
;; in __FILE__
load-file-name: /Users/sylvain/.emacs.d/init.el
__FILE__ result is: /Users/sylvain/.emacs.d/init.el

So it seems that anyone installing their packages from their emacs loading process in init.el would hit the bug.

[syl20bnr]

Reported in issue https://github.com/rocky/emacs-load-relative/issues/6

[rocky]

What's the contents of .emacs.d/init.el? Or better, perhaps you whittle down init.el to the smallest amount of code that exhibits the problem and come up with an isolated set of example elisp files (I imagine you need init.el with some other elisp file) that shows off the problem with __FILE__? Thanks

[syl20bnr]

Let me cook one up, meanwhile all Spacemacs is doing is calling package-install function from a required file in init.el.

[syl20bnr]

Try with (in your init.el):

(setq package-archives '(("melpa" . "https://melpa.org/packages/")
                         ("org" . "http://orgmode.org/elpa/")
                         ("gnu" . "https://elpa.gnu.org/packages/")))
(package-initialize)
(package-install 'realgud)

[rocky]

Sadly this isn't enough for me to reproduce the problem. I tried running the above after first removing realgud, load-relative and for good measure loc-changes. Inside Vanilla emacs, things load fine.

Perhaps there is a way to isolate things a little more?

[syl20bnr]

That's weird the above snippet is literally what I had in init.el and nothing else. What is your emacs version ?

[syl20bnr]

Also your OS please.

[rocky]

"GNU Emacs 25.1.50.2 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8) of 2015-11-30"

[syl20bnr]

Here is mine and bug confirmed again with the above snippet.

This is GNU Emacs 24.5.1 (x86_64-apple-darwin15.0.0, X toolkit, Xaw3d scroll bars) of 2016-02-06 on MBA14

[syl20bnr]

Another interesting fact, if I eval the same code from init.el after Emacs has loaded then it works. So the issue is to install realgud before Emacs has finished loading in which case the load filename is reported to be init.el.

[bmag]

I've noticed the same as @syl20bnr. I created a file named temp.el inside of .emacs.d, with similar content:

(require 'package)
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(package-initialize)
(package-refresh-contents)
(package-install 'realgud)

Running M-x eval-buffer installs without errors, but running M-x load-file temp.el installs with errors. System:

GNU Emacs 24.5.2 (i686-pc-linux-gnu, GTK+ Version 3.10.8) of 2015-06-19

[rocky]

Ok. Using M-x load-file temp.el I am now able to reproduce this problem. And the bug makes more sense to me although I'm not sure exactly how and where to address it.

This stuff is tricky so I welcome thoughts and comments on how to proceed here.

In short I wish Emacs supported something like __FILE__ and/or load-relative much as C/C++, Perl, Ruby, and nodejs do.

From the docstring comment for __FILE__ in load-relative.el:

Return the string name of file/buffer that is currently begin executed.

The first approach for getting this information is perhaps the most pervasive and reliable. But it the most low-level and not part of a public API, so it might change in future implementations. This method uses the name that is recorded by readevalloop of lread.c as the car of variable current-load-list.

Failing that, we use load-file-name which should work in some subset of the same places that the first method works. However load-file-name will be nil for code that is eval'd. To cover those cases, we try buffer-file-name which is initially correct, for eval'd code, but will change and may be wrong if the code sets or switches buffers after the initial execution.

As a last resort, you can pass in SYMBOL which should be some symbol that has been previously defined if none of the above methods work we will use the file-name value find via `symbol-file

See also the code further down .

So load-file-name is used and that is wrong here. Loadingload-relative.el setting a breakpoint in __FILE__ and stepping through and looking inside current-load-list I see the value:

((require . load-relative) (require . shell) (require . cl) "/home/rocky/.emacs.d/elpa/realgud-20160305.157/realgud/common/track-mode.el")

So possibly I can pick up the current file name later on in that list. But I'll have to study the Emacs C source code to see if this makes sense.

[syl20bnr]

That's interesting that you are reproducing it only by explicitly loading the file and not in init.el, maybe a side effect of a fix. If you have a possible solution we can test it before you commit it.

[rocky]

@syl20bnr may try to look at over the weekend. For those who want to experiment try changing https://github.com/rocky/emacs-load-relative/blob/master/load-relative.el#L142 so that it scans through the list picking up the first item that is a string, i.e. it skips those cons nodes.

[rocky]

@syl20bnr https://github.com/rocky/emacs-load-relative/commit/918a452dac1d2a5d7646c86af7178536f0e55f22 might address this.

Alternatively, I might broken things big time. We'll see.

[rocky]

@syl20bnr @bmag is this issue fixed?

[syl20bnr]

I'll check later today. Thank you for the fix.

[bmag]

@rocky doesn't seem to be fixed. Installing via load-file fails for me as before.

[rocky]

I looked at this adding

  (byte-compile-log-warning (format "XXX %s" current-load-list))
  (byte-compile-log-warning (format "XXX2 %s" load-path))

into load-relative's __FILE__ function and things are a big mess. The most promising approach I can think of right now is to look at the call stack to see if __FILE__ was called by byte-compile-file and if so see if I can get the filename from there.

I'm at a loss at how I'd do that though. I may ask on emacs-dev or stackoverflow for assistance.

[bmag]

As for examining the call stack, there's a solution on stack exchange that I've been using for a few months already in one of my packages. It can be used as:

(when (member 'byte-compile-file (function-stack))
  (do-stuff))

As for getting the file name, maybe the variable byte-compile-current-file can be used? I have no idea how reliable it is, but it seems to be let-bound in byte-compile-file to the value of the filename argument that byte-compile-file receives. I guess the file name is relative to default-directory.

[rocky]

I think/hope I have now found the problem, so I'd be grateful you test this yet one more time.

If this is it turns out to work, things are simpler than I had thought.

@bmag thanks for that information. Although I didn't go that route (this time), it definitely led me to what I believe was going wrong. And if I am correct, the problem was in premature compilation in require-relative-list.

Bullwinkle: Rocky - watch me pull a rabbit out of my hat! Rocky: What, that old trick never works. Bullwinkle: this time, for sure! ...

[bmag]

@rocky there are no compilation errors now, but (require 'realgud) throws an error:

Debugger entered--Lisp error: (invalid-function realgud-struct-field-setter)
  realgud-struct-field-setter("realgud-srcbuf-info" "cmdproc")
  byte-code(...)
  require(realgud-buffer-source "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/buffer/source")
  require-relative("buffer/source" "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/loc.elc" "realgud-buffer-")
  byte-code(...)
  require(realgud-loc "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/loc")
  require-relative("loc" "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/bp.elc" "realgud-")
  byte-code(...)
  require(realgud-bp "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/bp")
  require-relative("bp" "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/reset.elc" "realgud-")
  byte-code(...)
  require(realgud-reset "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/reset")
  require-relative("reset" "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/core.elc" "realgud-")
  byte-code(...)
  require(realgud-core "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/core")
  require-relative("core" "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/track-mode.elc" "realgud-")
  byte-code(...)
  require(realgud-track-mode "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud/common/track-mode")
  require-relative("./realgud/common/track-mode" "/home/bar/.emacs.d/elpa/realgud-20160305.157/realgud.elc" "realgud-")
  realgud:load-features()
  byte-code(...)
  require(realgud)
  eval((require (quote realgud)) nil)
  eval-expression((require (quote realgud)) nil)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!