A set of heuristics to improve large-file incremental change handling

[cjohansson]

Two heuristics:

1. Pressing return when rest of current line after cursor is white-space should move indexes forward: tokens, states, line-indents, imenu. Next line should get same line-indent as current line. When rest of current line after cursor is not white-space, trigger incremental processes instead.

2. Pressing backspace when rest of current line before cursor is white-space should move indexes backward: tokens, states, line-indents, imenu. When rest of current line before cursor is not whte-space, trigger incremental processes instead.

3. When user has done none-whitespace changes, determine unchanged previous position R, determine changed maximum position X, determine new buffer length L. Now do partial lex from R to X, if new states at X equals old states at X just move indexes with delta X, otherwise do partial lex of rest of buffer.

[cjohansson]

I reconsidered this given the problems caused by using defadvice to get rid of heuristic 1 and 2 and replace them with heuristic 3. That way we only need to rely on buffer-local hooks and variables. Heuristic 3 support is 80% finished now on develop

[cjohansson]

Resolved