Error during redisplay: (jit-lock-function 1) signaled (void-function poly-ein--narrow-to-inner)

[korayal]

I'm using spacemacs and I have been getting this error at the Messages buffer, while working on a buffer that has no relation to EIN. (init.el) Then I disabled/uninstalled it completely and still kept getting these errors even though there aren't any file that contains poly-ein--narrow-to-inner.

Error during redisplay: (jit-lock-function 3427) signaled (void-function poly-ein--narrow-to-inner)
Error during redisplay: (jit-lock-function 3927) signaled (void-function poly-ein--narrow-to-inner)
Error during redisplay: (jit-lock-function 2838) signaled (void-function poly-ein--narrow-to-inner)
Error during redisplay: (jit-lock-function 3338) signaled (void-function poly-ein--narrow-to-inner)
Error during redisplay: (jit-lock-function 3838) signaled (void-function poly-ein--narrow-to-inner)

And here's the debug log:

Debugger entered--Lisp error: (void-function poly-ein--narrow-to-inner)
  poly-ein--narrow-to-inner(1- #f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x159064277ab9>))
  apply(poly-ein--narrow-to-inner (1- #f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x159064277ab9>)))
  #f(compiled-function (&rest args2) #<bytecode 0x159064277aad>)(#f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x159064277ab9>))
  apply(#f(compiled-function (&rest args2) #<bytecode 0x159064277aad>) #f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x159064277ab9>) nil)
  rainbow-delimiters--propertize(17)
  font-lock-fontify-keywords-region(1 17 nil)
  font-lock-default-fontify-region(1 17 nil)
  font-lock-fontify-region(1 17)
  #f(compiled-function (beg end) #<bytecode 0x1feff61fffad>)(1 17)
  font-lock-ensure()
  intero-fontify-expression("bs :: ByteString")
  #f(compiled-function (beg end ty) #<bytecode 0x159064395fd1>)(11679 11681 "bs :: ByteString")
  #f(compiled-function (state reply) #<bytecode 0x159064b11515>)((:cont #f(compiled-function (beg end ty) #<bytecode 0x159064395fd1>) :source-buffer #<buffer DAccess.hs> :beg 11679 :end 11681) "bs :: ByteString\n")
  intero-network-call-sentinel(#<process backend> "connection broken by remote peer\n")

[dickmao]

Does M-: (featurep 'poly-ein) return t?

I suspect a poly-ein.elc byte-compiled version of poly-ein.el is still laying around. If you get rid of it, then EIN will no longer contaminate things. Thanks for trying out EIN. Sorry we couldn't make it work.

[korayal]

@dickmao it returns nil, in fact I made a search for ein on all emacs related folders, and couldn't find anything. I too suspect that this is about a byte-compiled file but maybe not from ein.

I'm actually going to try to make it work eventually. I heavily use EIN on other parts of my work, but it's so annoying that I'm getting an error about ein on a buffer that has no relation to that package.

[korayal]

Btw, when I enable ipython-notebook on spacemacs and add (require 'ein) inside dotspacemacs/user-config, the error goes away.

I'm not an expert on this, but it feels like a buffer that has nothing to do with ein, shouldn't get errors related to ein.

[dickmao]

Btw, when I enable ipython-notebook on spacemacs and add (require 'ein) inside dotspacemacs/user-config, the error goes away.

I thought you de-installed EIN. That you can just reenable it so easily suggests it's still there on the filesystem. Please get rid of all EIN-related files, and restart spacemacs. If you still get the error, then I'm afraid I cannot help you.

Yes, EIN does alter the global definitions of certain important functions such as syntax-ppss and jit-lock. We trigger certain behaviors only if we think we're in an EIN buffer. It's possible I botched that detection, but I can't know for sure because if EIN is only partially deinstalled, then all bets are off.

[korayal]

I thought you de-installed EIN. That you can just reenable it so easily suggests it's still there on the filesystem. Please get rid of all EIN-related files, and restart spacemacs. If you still get the error, then I'm afraid I cannot help you.

Uninstalling a package in Spacemacs is as easy as commenting it out from a list, so when we comment it out, it basically uninstalls and deletes any package related file (except the ones inside .cache folder, which I guess is being used only when we wanted to re-install, and I already deleted that one too)

The only addition I did was adding the line (require 'ein) to the part of Spacemacs configuration which would mean it's going to be applied at Emacs startup, rather than the startup of a layer (in Spacemacs the layer for EIN is called ipython-notebook) related buffer.

So, I'm pretty sure I've removed anything related to EIN. Maybe I should open a ticket in Spacemacs in parallel to this one.

[Gastove]

Hello :wave: I'm actually getting this identical error, and I don't use Spacemacs at all. (I have a roll-my-own Emacs setup.) EIN appears to have broken indent-according-to-mode basically everywhere, for me. For instance: trying to indent inrust-mode`:

Debugger entered--Lisp error: (void-function poly-ein--narrow-to-inner)
  poly-ein--narrow-to-inner(1- #f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x1c228a1>))
  apply(poly-ein--narrow-to-inner (1- #f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x1c228a1>)))
  #f(compiled-function (&rest args2) #<bytecode 0x1c22895>)(#f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x1c228a1>))
  apply(#f(compiled-function (&rest args2) #<bytecode 0x1c22895>) #f(compiled-function (&optional pos) "Parse-Partial-Sexp State at POS, defaulting to point.\nThe returned value is the same as that of `parse-partial-sexp'\nrun from `point-min' to POS except that values at positions 2 and 6\nin the returned list (counting from 0) cannot be relied upon.\nPoint is at POS when this function returns.\n\nIt is necessary to call `syntax-ppss-flush-cache' explicitly if\nthis function is called while `before-change-functions' is\ntemporarily let-bound, or if the buffer is modified without\nrunning the hook." #<bytecode 0x1c228a1>) nil)
  rust-paren-level()
  rust-mode-indent-line()
  indent-according-to-mode()
  indent-region-line-by-line(1 7472)
  indent-region(1 7472)
  (if (or (-contains\? orary/indent-sensitive-modes major-mode) (-filter (function derived-mode-p) orary/indent-sensitive-modes) orary/disable-auto-indent) nil (indent-region (point-min) (point-max)))
  (if orary/disable-clean-and-indent nil (if (or (-contains\? orary/indent-sensitive-modes major-mode) (-filter (function derived-mode-p) orary/indent-sensitive-modes) orary/disable-auto-indent) nil (indent-region (point-min) (point-max))) (if orary/disable-whitespace-cleanup nil (ethan-wspace-clean-all)))
  orary/clean-and-indent-buffer()
  funcall-interactively(orary/clean-and-indent-buffer)
  call-interactively(orary/clean-and-indent-buffer nil nil)
  command-execute(orary/clean-and-indent-buffer)

Removing ein does not effect this error for me; so far, the only thing that helps is removing the *.elc files from any mode effected by this. Which, in my case, is every major mode I have.

For what it's worth, fully deleting and re-installing everything at once, including EIN, does not address the problem.

[Gastove]

Ah:

(add-function
 :around (symbol-function 'syntax-ppss)
 (apply-partially #'poly-ein--narrow-to-inner #'1-))

(This is lisp/poly-ein.el; poly-ein--narrow-to-inner is defined on 212, advice follows.)

I don't have enough context into EIN and it's polymode integration to know what this is intended to do, but surely it should be gated or only added conditionally, no?

[dickmao]

but surely it should be gated or only added conditionally, no?

Yes, it should. Thank you.

I am confused by:

1. syntax-ppss gets add-function'ed by the above, which suggests poly-ein.el is read, and yet poly-ein--narrow-to-inner (also in poly-ein.el) doesn't get defined.
2. The decontamination involves recreating elc files from all packages. Had use-packageing ein caused the other packages' elc files to get regenerated with the add-function'ed syntax-ppss? Seems far out.

[Gastove]

@dickmao -- your confusions are a very good point, and I did a little more digging. I can't prove this for sure, but here's my current world-view on what's happening:

Tools like use-package often wind up evaluating a package at least once, whether or not it then is always require-ed. (In the case of use-package, this is to byte-compile packages and generate autoload hooks; I cannot speak to Spacemacs.) I believe this puts me in a position where:

1. There's an installation or update of ein; it mutates a global function.
2. After ein, another package is installed; it is then byte-compiled with the advised version of a global function.
3. After an exit and reload of Emacs, the symbols from ein are no longer in scope, but the advice remains in the byte-compiled major mode of an unrelated language.

If nothing else, this seems to me to explain why @korayal saw the problem go away by requiring ein.

I see you've already merged a fix! Very much appreciated. This is not my project, and, I'd like to offer two suggestions for your consideration:

The first is, would it be possible to replace your add-function work with vanilla Emacs functions? That is -- instead of using the advice system to wrap these functions, could you simply define new functions that compose around them?

If not that, then: may I suggest the defadvice macro, which has the upshot of being more introspectable? What I mean is: with EIN's old use of add-function:

(add-function
 :around (symbol-function 'syntax-propertize)
 (apply-partially #'poly-ein--narrow-to-inner #'1-))

we see a help-buffer like so:

syntax-propertize is a compiled Lisp function in ‘syntax.el’.

(syntax-propertize ARG1)

Not documented.

[back]

defadvice "tells" Emacs about what's been done. I use this in my own configs to add, for instance, auto-save on other-window. Given this:

(defadvice other-window (before other-window-auto-save activate)
  (orary/auto-save-command))

The help for other-window looks like so:

other-window is an interactive compiled Lisp function in ‘window.el’.

It is bound to C-x o.

(other-window COUNT &optional ALL-FRAMES)

:around advice: ‘ad-Advice-other-window’

Select another window in cyclic ordering of windows.
<snip>

It doesn't show up in the markdown, but that :around advice: block is a link, and can be used to at least know what advice has been added to the function:

other-window is an interactive compiled Lisp function in ‘window.el’.

It is bound to C-x o.

(other-window COUNT &optional ALL-FRAMES)

:around advice: ‘ad-Advice-other-window’

Select another window in cyclic ordering of windows.

I'm happy to help implement some of this, but again -- I still don't fully grok what's being accomplished here, so I would need guidance before I could really get rolling on a PR.

[dickmao]

2. After ein, another package is installed; it is then byte-compiled with the advised version of a global function.

I can't reproduce this. It perturbs me that the byte-compilation for foo.el depends on a runtime binding of code outside foo.el. I'd need a minimum reproducible example to make any headway.

[Gastove]

@dickmao yes, I agree this is frustrating. I also can't produce an exact MWE, because you've already merged a fix. (use-package has, to the best of my knowledge, no support for installing a package directly from github or at a specific version or commit. Not a criticism! I appreciate the fast fix. Just stating my difficulty in getting a MWE together.) use-package does a fair number of clever tricks to increase load time, but I suspect those tricks are key to what's happening here.

What is a LISP program but an amalgamation of state? My point in my last comment is: add-function is dangeous, because it changes state in a volatile way, hanging new functionality off of gobal symbols that might not exist later. That doesn't mean it's a bad technique -- far from it. Just a technique that requires caution. Changing global definitions in a language with a global namespace can have global consequences.

You've already addressed the issue, and I personally don't have an MWE available, so this can be closed as far as I'm concerned. That said, I've been reading the org-mode source, looking at how it uses advice, and I've found two things, either of which might also help here:

1. Wrap advice in eval-after-load, to be sure advice is only ever added when the required modules are in scope.
2. Use the local directive to add-function, effectively making the advice buffer-local. (More on this in the add-function documentation, but this seems like an excellent approach to me.)

Thanks for the time and work!

[dickmao]

2. Use the local directive to add-function, effectively making the advice buffer-local. (More on this in the add-function documentation, but this seems like an excellent approach to me.)

I add-function with local where I can (e.g., syntax-propertize-function). As I need to intercept out-of-band calls to the global defuns syntax-ppss and syntax-propertize, I don't currently believe I can localize those add-functions.

The following PR aims to prevent the contaminative byte-compilation by hamfistedly setting ad-default-compilation-action to never as soon as I advise syntax-ppss.