search

vim / vim

The official Vim repository
https://www.vim.org
Vim License
36.43k stars 5.44k forks source link

asciidoc.vim syntax speed with set cursorline or set relativenumber #2539

Open mikepqr opened 6 years ago

mikepqr commented 6 years ago

The asciidoc.vim syntax file that ships with vim has some severe performance issues that make scrolling with syntax highlighting on very difficult. As suggested in CONTRIBUTING I emailed the maintainer listed in the file, but he is no longer involved in asciidoc, and advised me to report it here.

As far as I can tell, the problem is with the asciidocLiteralParagraph syntax region.

With :syntime on and then :syntime report, I get results like this when scrolling down ~10 lines in an asciidoc file:

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  1.974639   2524   0       0.004338    0.000782  asciidocLiteralParagraph \(\%^\|\_^\s*\n\)\@<=\s\+\S\+
  0.949458   2524   0       0.010313    0.000376  asciidocMacroAttributes \(\\\@<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}\)\@<=\S\{-}\[
  0.692648   2524   191     0.002190    0.000274  asciidocMacroAttributes [\\0-9a-zA-Z]\@<!\w\(\w\|-\)*:\S\{-}\[
  0.424680   2524   0       0.005664    0.000168  asciidocDoubleDollarPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=\$\$..\{-}\(\$\$\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.414403   2524   0       0.001611    0.000164  asciidocTriplePlusPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=+++..\{-}\(+++\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.349319   2524   0       0.007933    0.000138  asciidocEmail      [\\.:]\@<!\(\<\|<\)\w\(\w\|[.-]\)*@\(\w\|[.-]\)*\w>\?[0-9A-Za-z_]\@!
  0.219631   2524   0       0.001481    0.000087  asciidocList       .\+\(:\{2,4}\|;;\)$
  0.167563   2524   0       0.002648    0.000066  asciidocURL        \\\@<!\<\(http\|https\|ftp\|file\|irc\):\/\/[^| \t]*\(\w\|\/\)
  0.165062   2461   0       0.001111    0.000067  asciidocHLabel     \(::\|;;\)\(\s\+\|\\$\)
  0.164349   2587   0       0.001767    0.000064  asciidocQuotedEmphasized2 \(^\|[| \t([.,=\]]\)\@<='\([' \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\('\([| \t)[\],.?!;:=]\|$\)\@=\)
  0.160388   2461   0       0.000705    0.000065  asciidocListLabel  \(:\{2,4}\|;;\)$
  0.146719   2713   126     0.000893    0.000054  asciidocQuotedEmphasized \(^\|[| \t([.,=\]]\)\@<=_\([_ \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\(_\([| \t)[\],.?!;:=]\|$\)\@=\)
  0.133182   2587   0       0.003897    0.000051  asciidocEntityRef  \\\@<!&[#a-zA-Z]\S\{-};
  0.130972   2587   0       0.002007    0.000051  asciidocQuotedUnconstrainedEmphasized \\\@<!__\S\_.\{-}\(__\|\n\s*\n\)
[snip]

That's ~2s to process a regular expression (that in the case of this file never matches), and a total of ~5s to scroll ~10 lines. Scrolling is extremely slow and as a result it's very difficult to do without significantly over/undershooting.

I can speed things up by doing set nocursorline, which is clearly part of the problem in this case (and others), but I get a more significant speedup by simply commenting out this line in asciidoc.vim:

syn region asciidocLiteralParagraph start=/\(\%^\|\_^\s*\n\)\@<=\s\+\S\+/ end=/\(^\(+\|--\)\?\s*$\)\@=/ contains=asciidocToDo

Can anyone help me understand what that regular expression does? Can it be made more efficient? Should it simply be removed from asciidoc.vim? (It's not obvious to me what this region does, so I'm not sure of the implications.)

chrisbra commented 6 years ago

syn region asciidocLiteralParagraph start=/(\%^|_^\s\n)\@<=\s+\S+/ end=/(^(+|--)\?\s$)\@=/ contains=asciidocToDo

I think it is trying to match literal parts (e.g. anything indented by at least one space). I don't know nothing about asciidoc, but I wonder whether 

 syn region asciidocLiteralParagraph start=/^\s\+\S\+/ end=/\(^\(+\|--\)\?\s*$\)\@=/ contains=asciidocToDo

would work better...

dpelle commented 6 years ago

Can you link to an ascidoc file which gives slow slow syntax highlighting?

I tried with this file https://raw.githubusercontent.com/powerman/asciidoc-cheatsheet/master/full.adoc but scrolling was very fast for me using vim-8.0.1428. By the way, which version of Vim do you use? Also, what is your 'regexpengine' setting? (:set regexpengine?). Perhaps your document has long lines and doing :set regexpengine=0 might speed up.

For me, ":synreport" results are completely different than those in the ticket description. This is what I get with the above document:

 TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  0.104560   2756   0       0.002345    0.000038  asciidocMacroAttributes \(\\\@<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}\)\@<=\S\{-}\[
  0.078079   2760   54      0.000396    0.000028  asciidocMacroAttributes [\\0-9a-zA-Z]\@<!\w\(\w\|-\)*:\S\{-}\[
mikepqr commented 6 years ago

Thanks. Here's an asciidoc that exhibits these problems on my system. It's just a gist of five paragraphs of lorem ipsum. Here's the first five lines of syntime report after scrolling through that file top to bottom by holding j with set cursorline. I get very similar results with set relativenumber.

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  2.453419   2401   0       0.028614    0.001022  asciidocLiteralParagraph \(\%^\|\_^\s*\n\)\@<=\s\+\S\+
  0.696462   2401   0       0.002257    0.000290  asciidocMacroAttributes [\\0-9a-zA-Z]\@<!\w\(\w\|-\)*:\S\{-}\[
  0.478283   2401   0       0.003248    0.000199  asciidocDoubleDollarPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=\$\$..\{-}\(\$\$\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.476732   2401   0       0.004739    0.000199  asciidocTriplePlusPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=+++..\{-}\(+++\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.462254   2401   0       0.002726    0.000193  asciidocMacroAttributes \(\\\@<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}\)\@<=\S\{-}\[

With set nocursorline and set norelativenumber the problem goes away: syntime report looks like this

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  0.045537   49     0       0.001617    0.000929  asciidocLiteralParagraph \(\%^\|\_^\s*\n\)\@<=\s\+\S\+
  0.014559   49     0       0.000564    0.000297  asciidocMacroAttributes [\\0-9a-zA-Z]\@<!\w\(\w\|-\)*:\S\{-}\[
  0.009441   49     0       0.000391    0.000193  asciidocDoubleDollarPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=\$\$..\{-}\(\$\$\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.009407   49     0       0.000429    0.000192  asciidocTriplePlusPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=+++..\{-}\(+++\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.009259   49     0       0.000359    0.000189  asciidocMacroAttributes \(\\\@<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}\)\@<=\S\{-}\[

Here's the output of :version

VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Jan  8 2018 13:56:05)
macOS version
Included patches: 1-1400
Compiled by Homebrew
Huge version without GUI.  Features included (+) or not (-):
+acl               +byte_offset       +comments          +digraphs          +file_in_path      +iconv             +linebreak         +mouse             +mouse_urxvt       +path_extra        +quickfix          +statusline        +terminal          +vertsplit         +windows           -xterm_save
+arabic            +channel           +conceal           -dnd               +find_in_path      +insert_expand     +lispindent        -mouseshape        +mouse_xterm       +perl              +reltime           -sun_workshop      +terminfo          +virtualedit       +writebackup
+autocmd           +cindent           +cryptv            -ebcdic            +float             +job               +listcmds          +mouse_dec         +multi_byte        +persistent_undo   +rightleft         +syntax            +termresponse      +visual            -X11
-autoservername    -clientserver      +cscope            +emacs_tags        +folding           +jumplist          +localmap          -mouse_gpm         +multi_lang        +postscript        +ruby              +tag_binary        +textobjects       +visualextra       -xfontset
-balloon_eval      +clipboard         +cursorbind        +eval              -footer            +keymap            -lua               -mouse_jsbterm     -mzscheme          +printer           +scrollbind        +tag_old_static    +timers            +viminfo           -xim
+balloon_eval_term +cmdline_compl     +cursorshape       +ex_extra          +fork()            +lambda            +menu              +mouse_netterm     +netbeans_intg     +profile           +signs             -tag_any_white     +title             +vreplace          -xpm
-browse            +cmdline_hist      +dialog_con        +extra_search      -gettext           +langmap           +mksession         +mouse_sgr         +num64             +python            +smartindent       -tcl               -toolbar           +wildignore        -xsmp
++builtin_terms    +cmdline_info      +diff              +farsi             -hangul_input      +libcall           +modify_fname      -mouse_sysmouse    +packages          -python3           +startuptime       +termguicolors     +user_commands     +wildmenu          -xterm_clipboard
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H   -DMACOS_X -DMACOS_X_DARWIN  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: clang   -L. -fstack-protector -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib  -L/usr/local/lib -o vim        -lncurses -liconv -framework AppKit   -mmacosx-version-min=10.12 -fstack-protector-strong -L/usr/local/lib  -L/usr/local/Cellar/perl/5.26.1/li
b/perl5/5.26.1/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -F/usr/local/opt/python/Frameworks -framework Python   -lruby.2.5.0 -lobjc

:set regexpengine is already =0.

Do you have set cursorline? Disabling that option seems to be an effective workaround, but it's a workaround, not a fix.

This seems performance issue makes it impossible to use for a bundled syntax file and a supported configuration option together. I know asciidoc is a relatively obscure format, but people have seen the same thing with the bundled syntax files for much more common formats, e.g. golang.

chrisbra commented 6 years ago

Have you tried the change I proposed?

mikepqr commented 6 years ago

Sorry, yes, I forgot to reply! Thank you for the suggestion. I believe you're correct in the intent of that syntax rule, and simplifying it as you suggest (or just doing autocmd syntax asciidoc syntax clear asciidocLiteralParagraph) does give a factor of ~2 speedup, which is great, thank you!

But it's still very slow, as the next ~10 or so rules each take up half a second each or so to scroll a ~50 line file. In my original post, I wrote

I can speed things up by doing set nocursorline, which is clearly part of the problem in this case (and others), but I get a more significant speedup by simply commenting out this line in asciidoc.vim:

But this is wrong. Sorry for the confusion. The performance issue seems to be caused by an interaction between set cursorline or set relativenumber and many/most of the rules in the bundled asciidoc syntax file.

For now I'm just going to set nocursorline, which is a shame, but disabling that one option speeds up those syntime report results by a factor of 10-1000 and makes the UI more usable.

jamessan commented 6 years ago

I see a few lookbehind assertions (\@<! and \@<=) in the syntax report. Those are known to be slower in the new regexp engine. However, they can commonly be sped up by bounding how far back they'll look. For example, \@1<= will only look back one byte. It's worth seeing if appropriate use of those bounds can help.

chrisbra commented 6 years ago

Please try out this commit: https://github.com/chrisbra/vim/commit/b90ea03579f1ca8cb4b48aa7ebbdfdff42ba0db3 It limits the negative lookbehind assertions to 10 bytes.

mikepqr commented 6 years ago

Thanks! But no joy. Same deal: slow with cursorline on, fine without.

syntime report with chrisbra/vim@b90ea03 and set cursorline

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  1.009018   1872   0       0.004509    0.000539  asciidocTriplePlusPassthrough \\\@10<!\(^\|[^0-9a-zA-Z$]\)\@<=+++..\{-}\(+++\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  1.001048   1872   0       0.002795    0.000535  asciidocDoubleDollarPassthrough \\\@10<!\(^\|[^0-9a-zA-Z$]\)\@<=\$\$..\{-}\(\$\$\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.514312   1872   0       0.000794    0.000275  asciidocMacroAttributes [\\0-9a-zA-Z]\@10<!\w\(\w\|-\)*:\S\{-}\[
  0.347771   1872   0       0.002307    0.000186  asciidocMacroAttributes \(\\\@10<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}\)\@<=\S\{-}\[
  0.293259   1872   0       0.001028    0.000157  asciidocEmail      [\\.:]\@10<!\(\<\|<\)\w\(\w\|[.-]\)*@\(\w\|[.-]\)*\w>\?[0-9A-Za-z_]\@!
  0.188125   1872   0       0.000383    0.000100  asciidocList       .\+\(:\{2,4}\|;;\)$
  0.142001   1872   0       0.000243    0.000076  asciidocHLabel     \(::\|;;\)\(\s\+\|\\$\)
  0.141810   1872   0       0.000302    0.000076  asciidocListLabel  \(:\{2,4}\|;;\)$
  0.137126   1872   0       0.000572    0.000073  asciidocURL        \\\@10<!\<\(http\|https\|ftp\|file\|irc\):\/\/[^| \t]*\(\w\|\/\)
  0.106591   1872   0       0.002028    0.000057  asciidocQuotedAttributeList \\\@10<!\[[a-zA-Z0-9_-][a-zA-Z0-9 _-]*\][+_'`#*]\@=
  0.105782   1872   0       0.000243    0.000057  asciidocQuotedDoubleQuoted \(^\|[| \t([.,=\]]\)\@<=``\([` \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\(''\([| \t)[\
[...]

syntime report with chrisbra/vim@b90ea03 and set nocursorline

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  0.013715   20     0       0.001572    0.000686  asciidocTriplePlusPassthrough \\\@10<!\(^\|[^0-9a-zA-Z$]\)\@<=+++..\{-}\(+++\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.012426   20     0       0.001065    0.000621  asciidocDoubleDollarPassthrough \\\@10<!\(^\|[^0-9a-zA-Z$]\)\@<=\$\$..\{-}\(\$\$\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
  0.007319   20     0       0.000633    0.000366  asciidocMacroAttributes [\\0-9a-zA-Z]\@10<!\w\(\w\|-\)*:\S\{-}\[
  0.004775   20     0       0.000428    0.000239  asciidocMacroAttributes \(\\\@10<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}\)\@<=\S\{-}\[
  0.003489   20     0       0.000262    0.000174  asciidocEmail      [\\.:]\@10<!\(\<\|<\)\w\(\w\|[.-]\)*@\(\w\|[.-]\)*\w>\?[0-9A-Za-z_]\@!
  0.002664   20     0       0.000268    0.000133  asciidocList       .\+\(:\{2,4}\|;;\)$
  0.002067   20     0       0.000252    0.000103  asciidocHLabel     \(::\|;;\)\(\s\+\|\\$\)
  0.001932   20     0       0.000153    0.000097  asciidocListLabel  \(:\{2,4}\|;;\)$
  0.001646   20     0       0.000120    0.000082  asciidocURL        \\\@10<!\<\(http\|https\|ftp\|file\|irc\):\/\/[^| \t]*\(\w\|\/\)
  0.001427   20     0       0.000130    0.000071  asciidocMacroAttributes \\\@10<!\[\{3}\(\w\|-\|_\|:\|\.\)\+
  0.001391   20     0       0.000242    0.000070  asciidocQuotedUnconstrainedMonospaced [\\+]\@10<!++\S\_.\{-}\(++\|\n\s*\n\)
  0.001373   20     0       0.000147    0.000069  asciidocMacroAttributes \\\@10<!<<"\{-}\(\w\|-\|_\|:\|\.\)\+"\?,\?
[...]