BIG speed problem for non-small outputs in *R*

[mmaechler]

Look at our own testing code https://github.com/emacs-ess/ESS/blob/master/etc/slow-long-output.R

and notice that we are back to a very bad state... similarly to what we had in Dec 2013, and Vitalie fixed then.

Also - I think differently than back then - when the *R* buffer already contains some 10'000 lines, almost every print(.) becomes relatively (or absolutely) slow.

[mmaechler]

Turning off font-locking in "iESS" aka *R* should help... but now have a dozen such options... is there a quick "turn off all font-locking in iESS" ?

Note: Rstudio has practically no fontification either in R and they are considerably faster with the examples I've tried.

[vspinu]

That's strange. I haven't been doing anything substantial on the inferior side in the past 6 months. Have no idea what went wrong. Will check it up.

[MilkWasABadChoice]

I'm seeing substantial slowdowns in my R buffer as well, and at least for me, it's coming from inferior-ess-fontify-region call out to (compilation--ensure-parse). I wrote a quick benchmark A/B test:

(defun run-ess-profiling-actions () "Execute a trivial R command purely for the side-effect of generating output in the R buffer that will trigger fontification." (dotimes (i 5) (rickb-send-string-to-R "sapply(rows,class)") (sleep-for 1)))

(progn (elp-instrument-function 'inferior-ess-input-sender) (elp-instrument-function 'jit-lock-function) (elp-instrument-function 'font-lock-fontify-region) (run-ess-profiling-actions) (elp-results) (elp-restore-all))

Here's the results:

Using stock ess-20151016.923:

Function Name Call Count Elapsed Average jit-lock-function 30 5.7617650000 0.1920588333 font-lock-fontify-region 30 5.7609949999 0.1920331666 inferior-ess-input-sender 5 1.9449049999 0.3889809999

Disabling compilation--ensure-parse call (in lisp/ess-inf.el line 409):

Function Name Call Count Elapsed Average inferior-ess-input-sender 5 0.007901 0.0015802 jit-lock-function 25 0.006145 0.0002458 font-lock-fontify-region 25 0.0054569999 0.0002182799

My workaround is just to disable that compilation--ensure-parse call – avoids the CPU spike & keeps reasonably interactive performance, and whatever that call does, I don't miss it :/

(I am using rainbow-mode/delimeters and other UI-heavy extensions, so this might be hitting me harder than those with less glitzy UI setups.)

[mmaechler]

Thank you, (I've added emphasis to your crucial "fix"). Looking forward to my colleagues' opinion on the (compilation--ensure-parse ....) call and its slowness.

[mmaechler]

Another version of the same bug: I have been debugging (R calling C code); adding Rprintf() statements in the C code. Because of a non-convergence, I quickly produced a foo.Rout file of almost 2 Gigabytes. Now, I've opened that in Emacs -> which switches to 'ess-transcript' mode and wants to fontify that whole buffer.... and I have to quickly try to kill this (emacs) process, because it basically locks the whole emacs ... and also a big part of the computer's responsiveness.

I think ESS should occasionally check sizes of R and *.Rout buffers and suspend font-locking such buffers entirely as soon as they have more than (say) 10'000 lines.

[lionel-]

maybe only suspend idle font-locking, but still allow JIT font-locking? I have only a very limited understanding of those things so this may not make sense.

[vspinu]

and whatever that call does, I don't miss it :/

It makes your error highlighting work such that you can M-g n quickly to the error location.

[vspinu]

I cannot find an example of code that would produce compilation problems that were the reason for that compilation--ensure-parse line. The problems were occurring mostly on remotes.

Given that the highlighting problem is not as severe as this bug, I am commenting that line and will come back once I stumble across a reproducible example again. Will take into account the speed problem then as well.

[mmaechler]

Thank you Vitalie (and @MilkWasABadChoice !).

[jason-morgan]

Thanks for the quick turnaround on this! I appreciate it.