emacs 100% cpu freeze when editing text in repl buffer

[vv111y]

It seems to happen after I paste text from another buffer, or it may be when I go between normal and insert state (using evil) under certain unknown circumstances.

• it is not another child process, it is emacs itself
• doesn't trigger debugger, so I can't use that
• using spacemacs on either mac or linux

info:

- OS: darwin (but happens on linux too)
- Emacs: 26.1
- Spacemacs: 0.300.0
- Spacemacs branch: develop (rev. 4b195ddfc)
- Graphic display: t
- Distribution: spacemacs
- Editing style: vim
- Completion: helm

only killing emacs and restarting works.

[non-Jedi]

I've seen this. I can reproduce consistently without pasting by doing the following with the IJulia kernel.

]activate ~/repos/ChemicalEngineeringToolbox.jl
]st

It freezes up before any letters are actually typed and before the prompt changes from In [2] to (v1.1) pkg>.

From ps auxf:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
adam     17944 64.0  2.6 611624 214252 ?       Rsl  11:30   4:26 emacs --daemon
adam     17962  0.0  0.1  18172  8516 ?        Ss   11:30   0:00  \_ /usr/bin/aspell -a -m -B --encoding=utf-8
adam     18081  0.1  0.3 364520 31760 ?        Ssl  11:32   0:00  \_ /usr/bin/emacs-26.1 -Q -batch -L /home/adam/.dotfiles/emacs-packages/emacs-zmq/ -l /home/adam/.dotfiles/emacs-packages/emacs-zmq/zmq.elc --eval (zmq--init-subprocess nil)
adam     18086  3.7  3.0 737456 240436 pts/3   Ssl+ 11:32   0:10  \_ /usr/bin/julia -i --startup-file=yes --color=yes /home/adam/.julia/packages/IJulia/4UizY/src/kernel.jl /run/user/1000/jupyter/emacs-kernel-i8i3Od.json

The weird part is that htop shows the emacs CPU as 100% as described above but ps says it's some number less than that.

[nnicandro]

Yeah I have had some issues with pasting in the REPL that I have been working on. I was getting freezes when I tried to undo a paste, but that should be fixed in 9484735c0651a412ad6e2a4d6628c740feda1419. I've always been able to C-g from the freeze though.

@non-Jedi it might have to do with the package prompt ] interfering with syntax parsing. Are you using an emacs-jupyter newer than 6eec94a97fe2e51f2f1a0e5157dceb6e0b05ef40? Also are saying that ]activate ~/repos/ChemicalEngineeringToolbox.jl runs but then after you type ]st RET everything freezes?

As a general tactic for attempting to un-freeze Emacs you can go to a shell and do

pkill -P <PID> -SIGUSR2

where <PID> is the process ID of the frozen Emacs. Try to do this a few times to see if you can get Emacs to spit out a backtrace. See debug-on-event.

[non-Jedi]

I'm using an older commit. Will upgrade and retest. And yes, the activate command runs successfully, and then emacs freezes as soon as you type ]. Sending a SIGUSR2 signal hasn't had any effect in the past.

EDIT: Still freezes after updating, but now it seems to be throwing an error. Will see if I can get a debugger traceback on it.

EDIT2: Backtrace below. Let me know if I should create a new issue since it now sounds like this is a separate problem. My issue is that "ChemicalEngineeringToolbox" won't fit in the margin I guess.

Debugger entered--Lisp error: (wrong-type-argument wholenump -22)
  make-string(-22 32)
  (concat (make-string (- jupyter-repl-prompt-margin-width (length str)) 32) str)
  (propertize (concat (make-string (- jupyter-repl-prompt-margin-width (length str)) 32) str) 'fontified t 'font-lock-face face)
  (list '(margin left-margin) (propertize (concat (make-string (- jupyter-repl-prompt-margin-width (length str)) 32) str) 'fontified t 'font-lock-face face))
  jupyter-repl--prompt-display-value("(ChemicalEngineeringToolbox) pkg> " ((:foreground "magenta3") jupyter-repl-input-prompt))
  (propertize " " 'display (jupyter-repl--prompt-display-value str (or face 'jupyter-repl-input-prompt)))
  (overlay-put ov 'after-string (propertize " " 'display (jupyter-repl--prompt-display-value str (or face 'jupyter-repl-input-prompt))))
  (progn (overlay-put ov 'after-string (propertize " " 'display (jupyter-repl--prompt-display-value str (or face 'jupyter-repl-input-prompt)))))
  (if ov (progn (overlay-put ov 'after-string (propertize " " 'display (jupyter-repl--prompt-display-value str (or face 'jupyter-repl-input-prompt))))))
  (let ((ov (car (overlays-at (jupyter-repl-cell-beginning-position))))) (if ov (progn (overlay-put ov 'after-string (propertize " " 'display (jupyter-repl--prompt-display-value str (or face 'jupyter-repl-input-prompt)))))))
  jupyter-repl-cell-update-prompt("(ChemicalEngineeringToolbox) pkg> " ((:foreground "magenta3") jupyter-repl-input-prompt))
  jupyter-julia-update-prompt("(ChemicalEngineeringToolbox) pkg> " "magenta3")
  (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))
  (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5))))
  (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))
  (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3))))
  (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3)))))
  (save-excursion (goto-char beg) (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3))))))
  (progn (save-excursion (goto-char beg) (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3)))))))
  (if (= beg (jupyter-repl-cell-code-beginning-position)) (progn (save-excursion (goto-char beg) (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3))))))))
  (progn (if (= beg (jupyter-repl-cell-code-beginning-position)) (progn (save-excursion (goto-char beg) (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3)))))))) (funcall cl--cnm))
  (progn (progn (if (= beg (jupyter-repl-cell-code-beginning-position)) (progn (save-excursion (goto-char beg) (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3)))))))) (funcall cl--cnm)))
  (closure (t) (cl--cnm _type beg _end) "Change the REPL prompt when a REPL mode is entered." (progn (progn (if (= beg (jupyter-repl-cell-code-beginning-position)) (progn (save-excursion (goto-char beg) (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3)))))))) (funcall cl--cnm))))(#f(compiled-function (&rest cnm-args) #<bytecode 0x15ab1f1>) insert 124 125)
  apply((closure (t) (cl--cnm _type beg _end) "Change the REPL prompt when a REPL mode is entered." (progn (progn (if (= beg (jupyter-repl-cell-code-beginning-position)) (progn (save-excursion (goto-char beg) (let* ((temp (char-after))) (cond ((eql temp '93) (if (and (boundp 'blink-paren-function) blink-paren-function) (progn (setq last-command-event 91))) (let* ((jupyter-default-timeout jupyter-long-timeout) (pkg-prompt (jupyter-eval "import Pkg; Pkg.REPLMode.promptf()"))) (if pkg-prompt (progn (jupyter-julia-update-prompt (substring pkg-prompt 1 (1- (length pkg-prompt))) (aref ansi-color-names-vector 5)))))) ((eql temp '59) (jupyter-julia-update-prompt "shell> " (aref ansi-color-names-vector 1))) ((eql temp '63) (jupyter-julia-update-prompt "help?> " (aref ansi-color-names-vector 3)))))))) (funcall cl--cnm)))) #f(compiled-function (&rest cnm-args) #<bytecode 0x15ab1f1>) (insert 124 125))
  #f(compiled-function (&rest args) #<bytecode 0x3548a2d>)(insert 124 125)
  apply(#f(compiled-function (&rest args) #<bytecode 0x3548a2d>) insert (124 125))
  #f(compiled-function (arg &rest args) #<bytecode 0x35488e1>)(insert 124 125)
  apply(#f(compiled-function (arg &rest args) #<bytecode 0x35488e1>) (insert 124 125))
  jupyter-repl-after-change(insert 124 125)
  (cond ((= len 0) (jupyter-repl-after-change 'insert beg end)) ((and (= beg end) (not (= 0 len))) (jupyter-repl-after-change 'delete beg len)) ((= (- end beg) len) (if (and (= len 1) (get-text-property beg 'rear-nonsticky) (= end (jupyter-repl-cell-end-position))) (progn (remove-text-properties beg end '(rear-nonsticky))))))
  (progn (cond ((= len 0) (jupyter-repl-after-change 'insert beg end)) ((and (= beg end) (not (= 0 len))) (jupyter-repl-after-change 'delete beg len)) ((= (- end beg) len) (if (and (= len 1) (get-text-property beg 'rear-nonsticky) (= end (jupyter-repl-cell-end-position))) (progn (remove-text-properties beg end '(rear-nonsticky)))))))
  (if (save-excursion (goto-char beg) (jupyter-repl-cell-line-p)) (progn (cond ((= len 0) (jupyter-repl-after-change 'insert beg end)) ((and (= beg end) (not (= 0 len))) (jupyter-repl-after-change 'delete beg len)) ((= (- end beg) len) (if (and (= len 1) (get-text-property beg 'rear-nonsticky) (= end (jupyter-repl-cell-end-position))) (progn (remove-text-properties beg end '(rear-nonsticky))))))))
  (condition-case err (if (save-excursion (goto-char beg) (jupyter-repl-cell-line-p)) (progn (cond ((= len 0) (jupyter-repl-after-change 'insert beg end)) ((and (= beg end) (not (= 0 len))) (jupyter-repl-after-change 'delete beg len)) ((= (- end beg) len) (if (and (= len 1) (get-text-property beg 'rear-nonsticky) (= end (jupyter-repl-cell-end-position))) (progn (remove-text-properties beg end '(rear-nonsticky)))))))) ((debug error) (message "Jupyter error after buffer change: %S" err) nil))
  (progn (condition-case err (if (save-excursion (goto-char beg) (jupyter-repl-cell-line-p)) (progn (cond ((= len 0) (jupyter-repl-after-change 'insert beg end)) ((and (= beg end) (not (= 0 len))) (jupyter-repl-after-change 'delete beg len)) ((= (- end beg) len) (if (and (= len 1) (get-text-property beg 'rear-nonsticky) (= end (jupyter-repl-cell-end-position))) (progn (remove-text-properties beg end '(rear-nonsticky)))))))) ((debug error) (message "Jupyter error after buffer change: %S" err) nil)))
  (if (eq major-mode 'jupyter-repl-mode) (progn (condition-case err (if (save-excursion (goto-char beg) (jupyter-repl-cell-line-p)) (progn (cond ((= len 0) (jupyter-repl-after-change 'insert beg end)) ((and (= beg end) (not (= 0 len))) (jupyter-repl-after-change 'delete beg len)) ((= (- end beg) len) (if (and (= len 1) (get-text-property beg 'rear-nonsticky) (= end (jupyter-repl-cell-end-position))) (progn (remove-text-properties beg end '(rear-nonsticky)))))))) ((debug error) (message "Jupyter error after buffer change: %S" err) nil))))
  jupyter-repl-do-after-change(124 125 0)
  self-insert-command(1)
  funcall-interactively(self-insert-command 1)
  call-interactively(self-insert-command nil nil)
  command-execute(self-insert-command)

[nnicandro]

@non-Jedi Yes can you make a new issue. In the new issue can you mention how the Julia REPL displays that prompt. Does it truncate it or just display the whole prompt? I'm not actually sure how to fix this because if we just display the whole prompt then basically half of your screen will be the margin. An alternative would be to insert the prompt directly in the buffer without using the margin, but I think this would take a lot of refactoring. The nice thing about using the margin is that the whole contents of the REPL buffer is either code or output without any prompt strings.

[vv111y]

What a relief I'm not the only one. It does it with evil-surround too. no interrupt seems to work here, I have to kill it everytime. I'll see if I can poke around and find something. I may be spending A LOT of time in here

[nnicandro]

@vv111y I can reproduce the issue with evil-surround. It's a good starting point for investigating the problem further. Regarding pasting into the REPL, since this may be a different problem, can you provide a recipe for reproducing the freeze?

[nnicandro]

OK I've found the issue with evil-surround. It has to do with the fact that the buffer is narrowed whenever changing the surround characters. The functions responsible for marking parts of the buffer as cell code (for font lock purposes) need to look at the parts of the buffer outside of the narrowed region. So since the buffer is narrowed, these functions are unable to mark the newly inserted surround characters as cell code and this seems to be the cause of the freeze.

Also, the narrowing happens because we use the field text property to mark cell code and distinguish it from cell output. When changing the surround, evil narrows to the current field (the evil-narrow-to-field call in evil-motion-range).

[guilherme-salome]

I am also having a problem with the REPL buffer freezing. I have never been able to recover with multiple C-g or even pkill and always need to force quite Emacs. The issue seems to occur whenever editing text in the REPL buffer.

I am using emacs-jupyter version 20190223.305 and elpy, but not evil. I am running Emacs 26.1 on MacOS. The same issue was happening on an earlier emacs-jupyter version (20190215).

How can I help with this issue? Is there a way to interrupt whatever is causing the issue when typing on the REPL buffer? The Jupyter REPL is very helpful and it would be very useful to be able to use it in addition to the code-blocks.

[nnicandro]

@Salompas When does the freeze happen? Does it happen when you try to indent a line? When you press enter to start a new line? Compared to the built-in python-mode from python.el, does elpy change any font locking stuff, e.g. what are the values of font-lock-fontify-region-function and syntax-propertize-function when inside an elpy buffer. Also, in the REPL buffer what is the value of after-change-functions?

EDIT: Can you also see if any elpy related modes are enabled in the REPL buffer, e.g. looking at the enabled minor modes when you press C-h m in the REPL buffer.

[nnicandro]

@vv111y The latest master fixes the issue with evil-surround, but may also fix your other issue. Could you try it out and let us know.

[guilherme-salome]

@Salompas When does the freeze happen?

The freeze happens when I am typing in the REPL. I usually work with an org file opened and split screen with the REPL buffer. After switching to the REPL buffer, if I start typing something and then try to delete some part of it, then that is usually when it freezes. I've been using sympy lately and it happens very consistently when typing Python code.

Does it happen when you try to indent a line?

It does not happen when I try to indent a line, simply because I am typing line by line on the REPL, so there is no indentation going on.

When you press enter to start a new line?

Also no. It happens while typing text and then deleting characters before pressing RET. Say you type something like sympy.Matrix([1, 2])*sympy.Matrix([[3, 4]]) and then go back to the beginning of the line to change Matrix to something else, then it would freeze.

Compared to the built-in python-mode from python.el, does elpy change any font locking stuff, e.g. what are the values of font-lock-fontify-region-function and syntax-propertize-function when inside an elpy buffer. Also, in the REPL buffer what is the value of after-change-functions?

Inspecting these variables from the REPL buffer, I get:

• C-h v font-lock-fontify-region-function:

  

• C-h v syntax-propertize-function

  

• C-h v after-change-functions

  

Oh, I just remembered it also froze once when resizing the width of the buffer with enlarge-window-horizontally.

[nnicandro]

@Salompas Can you actually show the font-lock-fontify-region-function and syntax-propertize-function whenever you are in a normal python buffer, e.g. opening a .py file. Also if you are able to get the freeze consistently, can you provide a minimum number of steps to reproduce it.

[guilherme-salome]

Also if you are able to get the freeze consistently, can you provide a minimum number of steps to reproduce it.

I've been able to get it to freeze consistently (tested 4 times and froze on all) by:

1. Launch Emacs and create a new org file;
2. Create a jupyter-python code block and type print('hi') and execute with C-c C-c to get the REPL buffer started;
3. C-x 3 to split the screen, C-x o to the other buffer and C-x b to make the REPL buffer visible;
4. Type: from sympy import * and hit RET
5. Then type: symbols('a')*Matrix([1,2]) and without hitting RET move the cursor to the end of the line with C-e, then C-b until the cursor is sitting on the parenthesis ( of symbols, then hit M-delete to delete symbols and Emacs will freeze and CPU usage will go to 100%ish

[guilherme-salome]

@Salompas Can you actually show the font-lock-fontify-region-function and syntax-propertize-function whenever you are in a normal python buffer, e.g. opening a .py file.

I launched Emacs, created a .py file and inspected the variables again. Here's what I got back:

• font-lock-fontify-region-function:

  

• syntax-propertize-function:

  

[nnicandro]

@Salompas OK, I've been able to reproduce this freeze. It is related to python.el font lock stuff and doesn't have anything to do with elpy. It looks like something in python-info-docstring-p which is called by python-font-lock-syntactic-face-function.

EDIT: Scratch that, the issue with python-info-docstring-p is actually another issue that I found.

[nnicandro]

@Salompas Should be fixed on latest master.

[guilherme-salome]

@Salompas Should be fixed on latest master.

Seems to be working!!!! Thank you very much for fixing! I'll keep testing and post if I find other freezes.

[vv111y]

@dzop thanks, you are really on this. It's great to see you so active getting this in top shape.

I'll test it out

[vv111y]

so far so good. Will spend a little more time with it to be sure.

[nnicandro]

@vv111y Does the freeze continue to happen for you or can this issue be closed?

[vv111y]

Yes, thank you