Closed machsix closed 6 years ago
Hi,
See the document in Vim. :h :syn-sync :h :syn-sync-third
Is the issue resolved with the following settings?
:syntax sync minlines=10000
No. It doesn't. If I use G
to directly jump from start of file to the end of file, syntax highlighting still breaks. Interestingly, I found if I do :20000
, :40000
then G
, syntax highlighting won't break.
Is it possible to upload the file where the problem occurs to Gist etc?
Does this only happen for asciidoc files? It looks like a problem of the syntax file. Here is the output of :syntime
after doing the G
jump (and it noticeably takes a couple of seconds until the file is redrawn with syntax on correctly:
TOTAL ANZAHL MATCH LANGSAMST DURCHSCHN NAME MUSTER
0.200709 34712 0 0.005521 0.000006 asciidocTriplePlusPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=+++..\{-}\(+++\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
0.135432 34712 0 0.000061 0.000004 asciidocDoubleDollarPassthrough \\\@<!\(^\|[^0-9a-zA-Z$]\)\@<=\$\$..\{-}\(\$\$\([^0-9a-zA-Z$]\|$\)\@=\|^$\)
0.088924 34712 0 0.000183 0.000002 asciidocMacroAttributes [\\0-9a-zA-Z]\@<!\w\(\w\|-\)*:\S\{-}\[
0.080844 34712 0 0.000268 0.000002 asciidocMacroAttributes \(\\\@<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}\)\@<=\S\{-}\[
0.063143 34712 0 0.000039 0.000002 asciidocEmail [\\.:]\@<!\(\<\|<\)\w\(\w\|[.-]\)*@\(\w\|[.-]\)*\w>\?[0-9A-Za-z_]\@!
0.041724 34712 0 0.000043 0.000001 asciidocURL \\\@<!\<\(http\|https\|ftp\|file\|irc\):\/\/[^| \t]*\(\w\|\/\)
0.040988 34712 0 0.000051 0.000001 asciidocLiteralParagraph \(\%^\|\_^\s*\n\)\@<=\s\+\S\+
0.039542 34712 0 0.000042 0.000001 asciidocList ^\s*\(\(\d\+\.\)\|\.\{1,5}\|\(\a\.\)\|\([ivxIVX]\+)\)\)\s\+
0.038525 34712 0 0.002541 0.000001 asciidocList ^\s*\(-\|\*\{1,5}\)\s
0.038258 34712 0 0.000066 0.000001 asciidocListNumber ^\s*\zs\(\(\d\+\.\)\|\.\{1,5}\|\(\a\.\)\|\([ivxIVX]\+)\)\)\ze\s\+
0.037079 34712 0 0.000032 0.000001 asciidocQuotedDoubleQuoted \(^\|[| \t([.,=\]]\)\@<=``\([` \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\(''\([| \t)[\],.?!;:=]\|$\)\@=\)
0.036827 34712 0 0.000096 0.000001 asciidocList .\+\(:\{2,4}\|;;\)$
0.036740 34712 0 0.000810 0.000001 asciidocQuotedBold \(^\|[| \t([.,=\]]\)\@<=\*\([* \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\(\*\([| \t)[\],.?!;:=]\|$\)\@=\)
0.036528 34712 0 0.001614 0.000001 asciidocQuotedEmphasized \(^\|[| \t([.,=\]]\)\@<=_\([_ \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\(_\([| \t)[\],.?!;:=]\|$\)\@=\)
0.036405 34712 0 0.000127 0.000001 asciidocQuotedAttributeList \\\@<!\[[a-zA-Z0-9_-][a-zA-Z0-9 _-]*\][+_'`#*]\@=
0.036352 34712 0 0.000189 0.000001 asciidocQuotedSingleQuoted \(^\|[| \t([.,=\]]\)\@<=`\([` \n\t]\)\@!\([^`]\|\n\(\s*\n\)\@!\)\{-}[^` \t]\('\([| \t)[\],.?!;:=]\|$\
0.035757 34712 0 0.000042 0.000001 asciidocMacroAttributes \\\@<!\[\{3}\(\w\|-\|_\|:\|\.\)\+
0.035313 34712 0 0.000034 0.000001 asciidocMacroAttributes \\\@<!<<"\{-}\(\w\|-\|_\|:\|\.\)\+"\?,\?
0.035204 34712 0 0.000029 0.000001 asciidocAttributeRef \\\@<!{\w\(\w\|[-,+]\)*\([=!@#$%?:].*\)\?}
0.035166 49658 0 0.000097 0.000000 asciidocCallout \\\@<!<\d\{1,2}>
0.034688 34712 0 0.000076 0.000001 asciidocQuotedEmphasized2 \(^\|[| \t([.,=\]]\)\@<='\([' \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\('\([| \t)[\],.?!;:=]\|$\)\@=\)
0.034575 34712 0 0.000044 0.000001 asciidocMacroAttributes \\\@<!\[\{2}\(\w\|-\|_\|:\|\.\)\+,\?
0.034524 34712 0 0.000042 0.000001 asciidocQuotedUnconstrainedMonospaced [\\+]\@<!++\S\_.\{-}\(++\|\n\s*\n\)
0.034100 34712 0 0.000136 0.000001 asciidocQuotedMonospaced2 \(^\|[| \t([.,=\]]\)\@<=`\([` \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\(`\([| \t)[\],.?!;:=]\|$\)\@=\)
0.033803 34712 0 0.000076 0.000001 asciidocQuotedMonospaced \(^\|[| \t([.,=\]]\)\@<=+\([+ \n\t]\)\@!\(.\|\n\(\s*\n\)\@!\)\{-}\S\(+\([| \t)[\],.?!;:=]\|$\)\@=\)
0.031844 34712 0 0.000135 0.000001 asciidocTable_OLD ^\([`.']\d*[-~_]*\)\+[-~_]\+\d*$
0.029696 34712 0 0.000038 0.000001 asciidocAdmonition ^\u\{3,15}:\(\s\+.*\)\@=
0.026437 34712 0 0.000091 0.000001 asciidocHLabel \(::\|;;\)\(\s\+\|\\$\)
0.026341 34712 0 0.000041 0.000001 asciidocListLabel \(:\{2,4}\|;;\)$
0.024397 34712 0 0.000035 0.000000 asciidocQuotedUnconstrainedBold \\\@<!\*\*\S\_.\{-}\(\*\*\|\n\s*\n\)
0.023728 34712 0 0.000082 0.000000 asciidocMacroAttributes \\\@<!(\{2,3}
0.023711 34712 0 0.000307 0.000000 asciidocEntityRef \\\@<!&[#a-zA-Z]\S\{-};
0.023680 34712 0 0.000059 0.000000 asciidocQuotedSuperscript \\\@<!\^\S\_.\{-}\(\^\|\n\s*\n\)
0.023046 34712 0 0.000029 0.000000 asciidocQuotedUnconstrainedEmphasized \\\@<!__\S\_.\{-}\(__\|\n\s*\n\)
0.022620 34712 0 0.000072 0.000000 asciidocListBullet ^\s*\zs\(-\|\*\{1,5}\)\ze\s
0.022468 34712 0 0.000041 0.000000 asciidocLineBreak [ \t]+$
0.022089 34712 9888 0.000042 0.000000 asciidocOneLineTitle ^=\{1,5}\s\+\S.*$
0.021861 34712 0 0.000028 0.000000 asciidocQuotedSubscript \\\@<!\~\S\_.\{-}\(\~\|\n\s*\n\)
0.019946 34712 34712 0.000794 0.000000 asciidocHLabel ^\s*
0.016853 34712 34712 0.000079 0.000000 asciidocListLabel ^\s*
0.012947 34712 0 0.000029 0.000000 asciidocTwoLineTitle ^[^. +/].*[^.]\n[-=~^+]\{3,}$
0.010380 34712 0 0.000060 0.000000 asciidocFilterBlock ^\w*\~\{4,}$
0.008155 34712 0 0.000028 0.000000 asciidocOpenBlockDelimiter ^--$
0.006011 34712 0 0.000042 0.000000 asciidocPagebreak ^<\{3,}$
0.004880 34712 7473 0.000021 0.000000 asciidocListingBlock ^-\{4,}$
0.004618 34712 0 0.000030 0.000000 asciidocPassthroughBlock ^+\{4,}$
0.004595 34712 0 0.000028 0.000000 asciidocExampleBlockDelimiter ^=\{4,}$
0.004416 34712 0 0.000031 0.000000 asciidocSidebarDelimiter ^*\{4,}$
0.004339 34712 0 0.000027 0.000000 asciidocRuler ^'\{3,}$
0.004324 34712 0 0.000026 0.000000 asciidocCommentBlock ^/\{4,}$
0.004167 34712 0 0.000027 0.000000 asciidocQuoteBlockDelimiter ^_\{4,}$
0.004044 22419 7473 0.000023 0.000000 asciidocListingBlock ^-\{4,}$
0.003801 34712 0 0.000027 0.000000 asciidocLiteralBlock ^\.\{4,}$
0.003168 34712 0 0.000026 0.000000 asciidocBlockTitle ^\.[^. \t].*[^-~_]$
0.003144 34712 0 0.000025 0.000000 asciidocCommentLine ^//\([^/].*\|\)$
0.003084 34712 0 0.000026 0.000000 asciidocListContinuation ^+$
0.003006 34712 0 0.000027 0.000000 asciidocAttributeEntry ^:\w
0.002830 34712 0 0.000019 0.000000 asciidocIdMarker ^\$Id:\s
0.002779 34712 0 0.000029 0.000000 asciidocBackslash \\
0.002683 34712 0 0.000053 0.000000 asciidocAttributeList ^\[[^[ \t].*\]$
0.002393 34712 0 0.000021 0.000000 asciidocTableBlock2 ^!=\{3,}$
0.002110 34712 0 0.000027 0.000000 asciidocTableBlock ^|=\{3,}$
1.761739 2154797
You might try to contact the syntax script maintainer and see if he knows of any settings to have it perform better.
This problem doesn't only happen for asciidoc file. Fortran code has the same problem. I uploaded a sample to here. With synctime report
, I found fortranConstructName
took more than 1s. If I set fortran_more_precise=0
, the the issue would disappear. Another workaround is to set re=1
.
But I didn't find the same problem with vim7.4
well, as said before, try contacting the syntax script maintainer and see if he has an idea how to improve the regexes there.
Consider re-opening this issue. At python-mode/python-mode#890, I voluminously document a similar issue. Under prior versions of Vim, syntax highlighting could always be restored at any arbitrary point of any arbitrarily large file by issuing some combination of the :syn sync fromstart
, :syn sync minlines=[insert small integer here]
, and/or :syn on
Ex commands.
Under Vim 8.0, however, none of these Ex commands actually do anything when editing large files; syntax highlighting remains fundamentally broken. Something appears to have gone awry in the Vim codebase itself – regardless of the time and space complexity of regular expressions in third-party syntax scripts. At the very least, Vim should supply an explicit warning, error, or other :message
as to why exactly it's refusing to perform syntax highlighting in a seemingly non-deterministic manner for large files.
same here. Please consider reopen it and try to fix this as this is really a bad experience for vim user.
Please tell me the setting value of your 'redrawtime'.
:set redrawtime?
Even if you set 'redrawtime' to 10000, will your problem occur?
:set redrawtime=10000
Thanks for suggesting the fix. I did another hack with
syntax sync fromstart
it starts working, although it's a little bit slow. I will also try your suggestion when it breaks again.
I ran into this editing large php files after updating from 8.0.0386 to 8.0.1298 under Gentoo, and can actually reproduce it reliably.
Oddly enough, attempting ":syntax sync fromstart" causes a failure with "Vim: Caught deadly signal SEGV" for me. However, as far as I can tell so far, changing that redrawtime from the default of 2000 to 10000 as suggested by h-east appears to correct this. I'll post back if I find otherwise, but so far so good.
Unlike wanchaol but like tlathm, :syntax sync fromstart
fails to suffice (as previously noted). Thankfully, like both h-east and tlathm, :set redrawtime=10000
does appear to suffice. And, like tlathm, I'm also on Gentoo – which seems more than a little suspicious, frankly. Kudos for the zany workaround, @h-east. (My brainpan is now exploding.)
Can you offer any deep insight as to why this only recently became an issue? Would it be feasible for Vim itself to dynamically increase this setting as a function of buffer size, thus automatically granting larger files more leeway with respect to syntax highlighting redraws than smaller files? Alternately, would globally bumping the default from its current value of 2000 to 10000 be feasible?
This behavior (When the syntax highlight processing exceeds 'rederawtime', it is disabled) was added in Vim 8.0.0647 on 18 Jun 2017. And bugfixed Vim 8.0.1133 on 22 Sep 2017.
Here is a document. :h 'rdt
'redrawtime' 'rdt' number (default 2000)
[...]
For syntax highlighting the time applies per window. When over the
limit syntax highlighting is disabled until |CTRL-L| is used.
This is used to avoid that Vim hangs when using a very complicated
pattern.
I think that it rarely exceeds the default value of 'redrawtime' 2000 msec probably if the specifications of the PC in the last 5 or 6 years. It happens with huge file wether uses PC specifications is low or very complicated pattern in syntax highlighting.
would globally bumping the default from its current value of 2000 to 10000 be feasible?
I don't think it's necessary. If you want to ensure syntax highlighting, always set a large value for 'redrawtime'.
It happens with huge file wether uses PC specifications is low
The Raspberry Pi is a thing. Just sayin'. :man_shrugging:
My machine is tolerably recent, but I still hit this issue. It's a bit disconcerting that Vim now silently fails in edge cases with no discernable error reporting. Which is the principal issue for me.
I'm quite O.K. with shifting the burden of resolution onto the end user by requiring that the default redrawtime
be manually overriden. That's fine. This is Vim, after all. We're all adult grognards here. What I'm not so O.K. with is Vim's failure to either:
redrawtime
causes syntax highlighting to permanently fail for the current buffer. That's bad. Ideally, Vim should be able to repair this situation on its own (e.g., by scheduling a subsequent redraw, perhaps with a redrawtime
temporarily set to a larger threshold).Isn't there something we (by whom I mean: someone not me) can do to at least inform users when this sort of catastrophic event occurs? Silent-but-deadly failures are the worst possible failures.
or very complicated pattern in syntax highlighting.
Right. In my case, python-mode
appears to be the culprit. I've yet to notice syntax highlighting failures with other modes. That said, @tlathm appears to hit this in some sort of PHP mode as well. whatevas
I don't think it's necessary.
Fair enough. Laziness is a virtue that I live by. That said, underpowered systems and/or complex regexes aren't the only culprits here. Even a high-powered system under sufficient CPU load will probably choke in similar ways when highlighting sufficiently large buffers.
This might be more widespread of an issue than you suspect.
Well the warning could happen often. But it might be a good idea to warn, when the syntax pattern timed out. So here is a patch, that warns if the verbose setting is larger 5 https://github.com/chrisbra/vim/commit/fab78fac76f882cd87059c9a91aaf314c609bcd2
I suppose that people who read the hint to set 'verbose' to 5 are already rare. Let's just give a message when disabling syntax highlighting. We'll have to see if this is too annoying.
Hm, I suppose that works as well. However now there is no indication, which pattern caused this. I guess one has to check the output of :syntime
and guess. BTW: does :syntime
work correctly once syn_win->w_s->b_syn_slow
has been set?
Hm, I suppose that works as well. However now there is no indication, which pattern caused this. I guess one has to check the output of
:syntime
and guess. BTW: does:syntime
work correctly oncesyn_win->w_s->b_syn_slow
has been set?
The pattern is only one that went over the limit, the cause might very well be another one.
:syntime should work to show what happened until the timeout. Haven't tried it though. Setting 'redrawtime' to a low value and using CTRL-L a few times should show that.
-- He was not in the least bit scared to be mashed into a pulp Or to have his eyes gouged out and his elbows broken; To have his kneecaps split and his body burned away And his limbs all hacked and mangled, brave Sir Robin. "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\ \\ an exciting new programming language -- http://www.Zimbu.org /// \\ help me help AIDS victims -- http://ICCF-Holland.org ///
To summarize the issue, is it fair to say that it is caused by
redrawtime
The solution is:redrawtime
to a large value like 10000syntax sync fromstart
when the highlighting breaksFrom this version, the following message is output when the syntax highlight becomes disabled in excess of 'redrawtime'
.
'redrawtime' exceeded, syntax highlighting disabled
Just installed 8.1.0198 under Gentoo and tested that new warning. Excellent...tells you exactly what to change. Thanks!
Thanks for the prompt attention, @brammool. The new warning is perfect in its succinct perfection – referencing the newly introduced redrawtime
setting for the uninitiated (i.e., me).
Party confections for all! :tada:
This also happens in Version 8.1. I'm a FE developer, the file are as follows:
<template lang="jade">
div.cntr
...
</template>
<style lang="stylus">
.cntr
color: red;
....
</style>
<script>
export default {
props: {
...
},
methods: {
function foo () {}
}
}
</script>
This file has 906 lines. When i scroll down by <Ctrl-D>
or <Ctrl-F>
, the syntax will broke, not always, but often. I followed the instruction of @leycec & @wanchaol by doing syntax sync fromstart
, then the syntax is back.
I don't know how this works, but the hack actually fix the problem. If anyone know, pls tell me. And I hope that this hack could help someone has the same problem.
FWIW: bumped into this while editing a large Python file (more than 10k lines).
set re=1
, as suggested above, appears to fix this.
VIM - Vi IMproved 8.1 (2018 May 18, compiled Mar 11 2019 08:36:40)
macOS version
I bumped into this today when editing a 2500-line Julia test file (lots of syntax highlighting). set redrawtime=10000
seems to have fixed things.
This problem doesn't only happen for asciidoc file. Fortran code has the same problem. I uploaded a sample to here. With
synctime report
, I foundfortranConstructName
took more than 1s. If I setfortran_more_precise=0
, the the issue would disappear. Another workaround is to setre=1
. But I didn't find the same problem with vim7.4
set re=1 worked for me working with files larger than 15000 lines.
Thank you.
Both "re" and "syntax" worked, but now and then I still have issues.
For me, the only definitive solution was creating that "syntax sync" mentioned earlier but in a "after" directory. I faced myself struggling with this same issue couple years ago and now again after migrating to nvim.
PC nvim ➜ pwd
/home/rodhash/.config/nvim
PC nvim ➜ tree after
after
└── syntax
├── html.vim
└── markdown.vim
1 directory, 2 files
PC nvim ➜ cat after/syntax/markdown.vim
syntax sync minlines=10000
" syntax sync fromstart
I'm working with MD for couple hours now (among other things) and no issue so far thus no need to manually run the syntax sync. Apparently it fixes the issue on nvim as well.
ps: I do remember reading some similar approach in other place but can't remember to link here.
The following line solved my problem:
set re=0
What it does is that it tells vim to use new regular expression engine. It seems that using old regular expression in vim leads to syntax highlighting issues as it was mentioned here.
After my initial comment, I realized that there was some confusion in this thread, and two issues were actually being discussed in this thread:
The former is related to this thread. The latter was split into #10452
I also encounter this on my primary dev machine, which is running Arch Linux, and not macOS. I added the following to my `~/.vimrc`: ```vimscript set redrawtime=10000 set re=0 ``` Neither of these fixed the issue. This seems to happen, not only with large files, but with files that utilizes multiple syntaxes. @Tao-Quixote had a great example of this, where a file is using HTML, CSS/SASS, and JavaScript or TypeScript. Closing the buffer and reopening it typically reproduces the issue pretty effectively. But also, if you have multiple buffers open, switching your window to a different buffer and back again, will also reproduce the issue. So far, the only thing that works is manually running `:syntax sync fromstart`Running VIM 8.2
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled May 13 2022 20:56:28) macOS version - x86_64 Included patches: 1-4950 Compiled by Homebrew Huge version without GUI. Features included (+) or not (-): +acl +file_in_path +mouse_urxvt -tag_any_white +arabic +find_in_path +mouse_xterm -tcl +autocmd +float +multi_byte +termguicolors +autochdir +folding +multi_lang +terminal -autoservername -footer -mzscheme +terminfo -balloon_eval +fork() +netbeans_intg +termresponse +balloon_eval_term +gettext +num64 +textobjects -browse -hangul_input +packages +textprop ++builtin_terms +iconv +path_extra +timers +byte_offset +insert_expand +perl +title +channel +ipv6 +persistent_undo -toolbar +cindent +job +popupwin +user_commands -clientserver +jumplist +postscript +vartabs +clipboard +keymap +printer +vertsplit +cmdline_compl +lambda +profile +vim9script +cmdline_hist +langmap -python +viminfo +cmdline_info +libcall +python3 +virtualedit +comments +linebreak +quickfix +visual +conceal +lispindent +reltime +visualextra +cryptv +listcmds +rightleft +vreplace +cscope +localmap +ruby +wildignore +cursorbind +lua +scrollbind +wildmenu +cursorshape +menu +signs +windows +dialog_con +mksession +smartindent +writebackup +diff +modify_fname -sodium -X11 +digraphs +mouse -sound -xfontset -dnd -mouseshape +spell -xim -ebcdic +mouse_dec +startuptime -xpm +emacs_tags -mouse_gpm +statusline -xsmp +eval -mouse_jsbterm -sun_workshop -xterm_clipboard +ex_extra +mouse_netterm +syntax -xterm_save +extra_search +mouse_sgr +tag_binary -farsi -mouse_sysmouse -tag_old_static system vimrc file: "$VIM/vimrc" user vimrc file: "$HOME/.vimrc" 2nd user vimrc file: "~/.vim/vimrc" user exrc file: "$HOME/.exrc" defaults file: "$VIMRUNTIME/defaults.vim" fall-back for $VIM: "/usr/local/share/vim" Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DMACOS_X -DMACOS_X_DARWIN -g -O2 -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 Linking: clang -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/readline/lib -L/usr/local/lib -o vim -lm -lncurses -liconv -lintl -framework AppKit -L/usr/local/opt/lua/lib -llua5.4 -mmacosx-version-min=12.0 -fstack-protector-strong -L/usr/local/lib -L/usr/local/Cellar/perl/5.34.0/lib/perl5/5.34.0/darwin-thread-multi-2level/CORE -lperl -L/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin -lpython3.10 -framework CoreFoundation -lruby.3.1 -L/usr/local/Cellar/ruby/3.1.2/lib
This issue was already reported by somebody else on vim-use mailist. In short words, if a file is very large (more than 10k lines), the syntax highlighting will break if I use
:linenumber
or search to jump to another line. But it won't break if I keep using PageDown to jump. I need to revoke:syntax on
to get highlighting back.Minium vimrc to reproduce:
Even if I add
set synmaxcol=0
to the file, the problem will occur. Vim version info:Example of "large file" is provided in the thread vim-use mailist. BTW: this issue doesn't exist in vim74. Thanks for your help!