emacs-ess hangs with Emacs 28.1.

[maitra]

With the recent update of emacs to 28.1 (on Fedora, but this does not appear to be a Fedora issue), emacs hangs with emacs-ess installed. Here is what happpens:

Run "emacs --no-init-file" from a terminal app.

Actual results:

Emacs hangs with message "Loading /usr/share/emacs/site-lisp/site-start.d/ess-init.el (source)..." in the mode line. After some time, the whole machine hangs.

Removing emacs-ess and then running emacs "works" in that emacs does not hang and starrts up normally (with warnings for the lack of emacs-ess, etc).

Installing emacs-ess once again starts emacs, but with the following warning:

Warning (comp): /usr/share/emacs/site-lisp/ess/ess-r-package.el: Error: Wrong number of arguments (3 . 4) Disable showing Disable logging

However, I am not sure what this means. It seems that something may have changed with emacs-ess as it interacts with emacs.

Happy to provide more information, and to test.

Thanks for emacs-ess. I have used it forever!

[maxecharel]

hi; how have you installed ESS, and which version?

[maitra]

hi; how have you installed ESS, and which version?

Thanks! I installed emacs-ess and emacs-common-ess from the Fedora repositories. The versions are 18.10.2.

[D4N]

This is a duplicate of https://github.com/emacs-ess/ESS/issues/1097, which itself is a duplicate of #1085. We have to cherry-pick https://github.com/emacs-ess/ESS/commit/9cc5520e1998d03f5dec0fbb1fe71b7cdec38b65 to the Fedora package, as define-obsolete-function-alias is a macro taking three arguments starting with Emacs 28.

On this note, would it be possible to get a new stable release for Emacs 28 compatibility?

[D4N]

Ok, the patch from https://github.com/emacs-ess/ESS/commit/9cc5520e1998d03f5dec0fbb1fe71b7cdec38b65 is not enough, if you try my scratch build including that patch (https://koji.fedoraproject.org/koji/taskinfo?taskID=89831324), then it'll still nearly murder your system :-/.

[maitra]

Ok, the patch from 9cc5520 is not enough, if you try my scratch build including that patch (https://koji.fedoraproject.org/koji/taskinfo?taskID=89831324), then it'll still nearly murder your system :-/.

Hopefully, we can get a Emacs 28.1 compatible ess package.

[HinTak]

Oh, I filed this at redhat two days ago https://bugzilla.redhat.com/show_bug.cgi?id=2109745 , as it was brought on by the emacs 27 to 28 upgrade, and downgrading emacs worked around it. I didn't think to look at the ess upstream.

[HinTak]

There are some version etc info in my filing. FWIW, machine does not hang - it just spawns multiple child processes, and write stuff to /tmp . You can stop it by going to another terminal with "killall emacs". This is on fedora.

[HinTak]

Here was my description of the problem : "emacs-ess spawn multiple processes with ... --batch -l /tmp/emacs-async-comp-ess-custom-*.el" . I did not look for hangs, etc. Only emacs itself get stuck, and I have been able to stop it, twice, so technically it is simply not a hang. It is just busy doing multiple stuff, which one can kill off quite easily.

[HinTak]

I don't think it is the same ad the other two - related, maybe. And it is not a hang! It is just busy launching new child processes. It is easy enough to kill too, when it happens.

[HinTak]

Hangs is unresponsive etc. But all I saw is emacs spawning about 100+ copies of itself, because I noticed and kill it off early (and did it twice). I did not let it get to to the 1000+ 10000+ child processes and affecting everything stage.

[HinTak]

If you kill it off early and look at the leftover *.el pieces in /tmp, you might get a better idea of what needs to be done. This is directed towards the ess people.

[HinTak]

The resource exhaustion seems to be brought on by byte code compilation. Anybody looking at the leftover *.el pieces yet?

[HinTak]

I fixed a somewhat similar issue in the emacs 23 / 25 time frame. You don't have to know too much about lisp. If you have a script which can reliably / reproducibly causes emacs to go into a long spin, you just insert a few "got here location 1", "got here location 2" print statements etc until you get down to the line which triggers the problem. Then the immediate work around would be protecting the problematic line with "(emacs_major_version < 28) ...", and a longer term plan posting to emacs-devel asking the emacs people why that line causes a problem with newer emacs. The issue I had then was a script which process non-english localized (ie also non-unicode) encoded inputs. Changes within emacs caused it to go into transcoding over and over from external localised encoding to emacs' internal extended unicode encoding. There is a variable in emacs to set to do "raw input" to disable that transcoding. It looks a similar issue.

I don't use R any more - not for a few years - and have ess installed mostly as carry-over from the past, so just prefer not to spend the time on this.

[HinTak]

It should be simple enough to put a "(emacs_major_version < 28)...)" somewhere to get emacs 28 to function as as plain text editor, until a better answer is found.

[aranc23]

I just wanted to add some confirmation that emacs-ess with 28.1 (on fedora, using the fedora packages) will basically render a system unusable often incurring the wrath of the OOM killer which kills emacs and other components as well. I thought maybe it had to do with the native compilation (and it probably does) so I tried disabling or curbing native compilation to no avail. I don't know much about this feature, and I'm not an ess user per-se I just support people who are. If the spawned processes were kept to a reasonable number I don't think this would be an issue but I am not sure if that's under control of ess or not. I'd be happy to test any fixes but for now I think I will be downgrading emacs for affected users.

[HinTak]

I commented on my fedora bug filing, that I have the system monitor gnome extension on, so emacs (or other run-away processes) does not spawn too many before I noticed and killed it. It's still around a hundred copies of itself, by counting how many left-over *.el script it creates in/tmp. It is not too bad if you kill it early enough.

As I said above, fixing this (or finding a simple work-around conditioning on emacs major version) is not technically demanding - just needing a good amount of time and patience, to bisect a lisp script with a pair of print statements.

[torbjorn]

Im also waiting for this to be solved. Till it is I will either look at a godo way to downgrade emacs to 27 or swap from fedora to ubuntu

[juhp]

Can you not try using the snapshot in MELPA say? That appears to install for me at least.

[torbjorn]

It seems to work, at least used in a fedora 36 docker container.

[torbjorn]

Trying to install it in my desktop environment, in an emacs session started with -q, I get these messages and errors during install:

Leaving directory ‘/home/username/.emacs.d/elpa/ess-20220815.2020’

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-bugs-d.el at Sat Aug 20 20:54:18 2022
Entering directory ‘/home/username/.emacs.d/elpa/ess-20220815.2020/’

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-bugs-l.el at Sat Aug 20 20:54:18 2022

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-custom.el at Sat Aug 20 20:54:18 2022

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-gretl.el at Sat Aug 20 20:54:18 2022

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-help.el at Sat Aug 20 20:54:18 2022
ess-help.el:37:13: Error: Symbol’s chain of function indirections contains a loop: R-mode

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-inf.el at Sat Aug 20 20:54:18 2022
ess-inf.el:35:13: Error: Symbol’s chain of function indirections contains a loop: R-mode

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-jags-d.el at Sat Aug 20 20:54:18 2022

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-julia.el at Sat Aug 20 20:54:18 2022
ess-julia.el:332:35: Warning: reference to free variable
    ‘ess-r-mode-syntax-table’

In end of data:
ess-julia.el:367:4: Warning: the function ‘ess--setup-company’ is not known to
    be defined.
ess-julia.el:365:4: Warning: the function ‘ess--setup-auto-complete’ is not
    known to be defined.
ess-julia.el:363:4: Warning: the function ‘ess--setup-eldoc’ is not known to
    be defined.
ess-julia.el:71:33: Warning: the function ‘ess--foreground-command’ is not
    known to be defined.

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-mode.el at Sat Aug 20 20:54:18 2022

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-r-completion.el at Sat Aug 20 20:54:18 2022

In ess-r-eldoc-function:
ess-r-completion.el:74:24: Warning: ess-eldoc-docstring-format called with 3
    arguments, but accepts only 2

In end of data:
ess-r-completion.el:414:39: Warning: the function ‘ess-installed-packages’ is
    not known to be defined.

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-r-flymake.el at Sat Aug 20 20:54:18 2022

Compiling file /home/username/.emacs.d/elpa/ess-20220815.2020/ess-r-mode.el at Sat Aug 20 20:54:18 2022

In R-initialize-on-start:
ess-r-mode.el:650:72: Warning: reference to free variable
    ‘ess-format-command-alist’
ess-r-mode.el:650:72: Warning: assignment to free variable
    ‘ess-format-command-alist’

When I start emacs -q and do load-library with ess, I get the following:

Warning (comp): /usr/share/emacs/site-lisp/ess/ess-r-package.el: Error: Wrong number of arguments (3 . 4) Disable showing Disable logging

[HinTak]

Tried the latest ESS git HEAD against emacs 28, same problem, so I had a look, as I outlined above, so here is the finding: https://github.com/emacs-ess/ESS/issues/1222

long story short, the resource exhaustion seems to be coming from emacs 28's native compilation capability, which is a new feature in emacs 28 which fedora switched on by default.

[maitra]

Tried the latest ESS git HEAD against emacs 28, same problem, so I had a look, as I outlined above, so here is the finding: #1222

long story short, the resource exhaustion seems to be coming from emacs 28's native compilation capability, which is a new feature in emacs 28 which fedora switched on by default.

Thank you for your thorough look at this. So, in your view, how should we deal with this? I don't know if Fedora emacs packagers would be cool with switching off emacs 28's native compilation capability by default, so a ESS solution would perhaps be better. Thanks again!

[HinTak]

It is a new feature so documentation is a bit sparse. Nonetheless, there seems to be some tips about setting some lisp variables, disabling it on for all packages:

https://emacs.stackexchange.com/questions/71335/how-to-prevent-emacs-28-from-native-compiling-everything

Or per package:

https://www.emacswiki.org/emacs/GccEmacs

Here is the url for the official documentation on it: https://www.gnu.org/software/emacs/manual/html_node/elisp/Native_002dCompilation-Functions.html

Probably best to update the ess code to work correctly with native compilation, I think.

[HinTak]

And perhaps setting native-comp-debug would help fixing the ess code: https://www.gnu.org/software/emacs/manual/html_node/elisp/Native_002dCompilation-Variables.html

[juhp]

Does setting native-comp-speed or native-comp-async-jobs-number help at all?

[lionel-]

The original issue is a duplicate of #1085, so I'm closing this.

Let's discuss the native compilation issues in the new thread #1222.

[HinTak]

The warnings and the resource exhaustion (I don't call it a hang, as you can kill it by hand within a good part of a minute) are unrelated. How many people experience the resource exhaustion on non-fedora?