hangs with very long lines

[GoogleCodeExporter]

What steps will reproduce the problem?
1. open a file that have some very long line
2. move up and/orw down through the file with the arrow keys and other methods 
(such as jump to the beginning, end or file or to a substring that matches a 
search)
3. move up and or down with the arrow keys through some part of the files where 
there are very long lines

What is the expected output? 

Nothing special should happen. Navigating could be awkward because of the fact 
that the whole line doesn't even fit within the terminal, but it should never 
become slow in responding to keyword or mouse interaction

What do you see instead?

vim hanged, started consuming 100% cpu and stopped responding. After several 
minutes I had to kill it.

This is extremely critical as it may imply data loss.

What version of the product are you using? On what operating system?

Ubuntu 14.04

Please provide any additional information below.

Original issue reported on code.google.com by matteosi...@gmail.com on 22 Oct 2014 at 8:18

[GoogleCodeExporter]

I don't know how long a line must be to reproduce the issue. I was editing a 
large file (around 75 MB) which was a dump of a mysql database (.sql file)

Original comment by matteosi...@gmail.com on 22 Oct 2014 at 8:19

[GoogleCodeExporter]

Does this happen if you disable syntax highlighting? Is it reproducable when 
you start vim with vim -u NONE -N
Can you provide the Vim version? Do you have any plugins installed?

Original comment by chrisbr...@googlemail.com on 22 Oct 2014 at 9:03

[GoogleCodeExporter]

Have you tried editing this file with the LargeFile.vim plugin?  (see 
http://www.drchip.org/astronaut/vim/index.html#LARGEFILE)
I would first suspect syntax highlighting (like Christian), but folding, undo, 
backing up, swapfile generation, etc can also slow things down.

Original comment by drc...@campbellfamily.biz on 24 Oct 2014 at 8:03

[GoogleCodeExporter]

I guess it must be syntax highlight, though I haven't tried disabling it yet.
I was actually just reading the file without modifying it (I hadn't opened it 
in readonly, but I didn't do any modifications, I only moved the cursor up and 
down).

You shouldn't need a plugin in order to manage large files efficiently.

I'm not sure whether the problem here is large file or very long lines. If 
managing a very long line is more problematic than managing the same text split 
in smaller lines (whether it's because of syntax highlight or not), then there 
is something wrong.

If syntax highlight is the problem and it's unavoidable (i.e. there's no known 
algorithm to manage it without the cost growing with the size of the file), 
then a warning should be shown when it starts being a problem, allowing the 
user to switch it off. It is never acceptable that the interface (even if it is 
a text-based interface) stops responding.

Also, after several minutes without touching it, vim was still processing every 
single keystroke from minutes earlier. That is, if I had hit the up and down 
arrow keys several times, you would see the text scrolling up one line, then 
after several seconds or more than a minute, up one more line, and so on.
I'm not sure if all keystrokes are queued up and processed blindly regardless 
of wheter or not there are more keystrokes in the queue, but if that is the 
case (as it seems to be), then that's wrong too. If the program gets blocked 
processing a command, and after that, there are several consecutive keystrokes 
in the queue that are merely cursor movements, it should not process each one 
and display the result of each one, it should just sum up the movements and 
process one single move; e.g. if you have hit the up key 5 times and the down 
key 9 times, instead of processing every single keystroke and "render" the 
result for each one, it should just move down 4 lines and render the final 
result. Even if more complex sequences of commands are "queued" while rendering 
can't keep the pace, and they need to be processed one by one, the "rendering" 
should still be skipped unless it's necessary in order to process the next 
command (e.g. the position of the cursor needs to be known). Syntax highlight 
can *certainly* always be skipped.

Original comment by matteosi...@gmail.com on 25 Oct 2014 at 10:06

[GoogleCodeExporter]

It might also work better, if you switch the 're' engine since for some regular 
expressions the NFA engine is known to be slow with long lines (see issue 
https://code.google.com/p/vim/issues/detail?id=164).

Anyway, it would help if you could confirm, that it is the syntax highlighting 
and which one is slow (See the help at :h :syntime) or what else is causing 
this.

Original comment by chrisbr...@googlemail.com on 26 Oct 2014 at 7:48

[GoogleCodeExporter]

Can you check, if Vim 7.4.497 fixes your issue?

Original comment by chrisbr...@googlemail.com on 5 Nov 2014 at 6:39