Only update higlights from the first changed line

[alerque]

Work in progress for #5.

First mistake: VIM informs us of the first changed line relative to the window, not the buffer...

[alerque]

Okay fixed one bug (the documentation for the callback is wrong) so now we are only updating from the first place the highlights could possibly differ.

Now we need to figure out where to stop. Probably just stopping somewhere after 2x the window height in lines would work in most cases, but what if we keep map of line numbers that begin new block level syntax and refresh up to the next block boundary. We could even tell Rust what byte that is up to and save even passing the data back and forth past where we need it.

[alerque]

This needs refinement. Changing a line can actually affect lines before it, notably in the case of closing a block syntax.

A line
▒

So far L1 is just plain paragraph text. Okay, now on L2 enter ====== to get:

A line
======▒

Suddenly L1 has become a header.

We need to rewind and apply highlihting to the lowest of ⓐ the first line of the current block or ⓑ the first line of the block as it was before we changed the current line.

[fmoralesc]

Since we have information about the block, couldn't we look for the last block start event to get the offset of the line we should start reparsing from? I think we could also stop highlighting as soon as we find a new block. neovim's added highlights are supposed to move around on text changes, so I think it isn't actually necessary to reapply highlighting to the whole document after the current position. I haven't tested this hypothesis, though.

[alerque]

Since we have information about the block, couldn't we look for the last block start event to get the offset of the line we should start reparsing from?

Kind of. The caveat is that we can't just search the current structure for the start of the block, we need to find whatever is earlier, the current or previous block. This is notable when dealing with long running fenced divs for example, if you break the end tag ... or add it ... the syntax as far back as the start of the fence (which could be quite a few blocks back) needs to update.

I think we could also stop highlighting as soon as we find a new block. neovim's added highlights are supposed to move around on text changes, so I think it isn't actually necessary to reapply highlighting to the whole document after the current position. I haven't tested this hypothesis, though.

This is true, and does work. The same caveat applies though, adding or removing something needs to parse to the largest bit of the document that could have changed. Say a document has an ending close fenced div or fenced code black, and hundreds of lines earier you add or remove the start of the fence. That needs to re-highlight everything in the fenced area --- that was before or is now.

Basically I'm saying we need to retain state of blocks between passes. I think we can actually do this on the Rust side, but it needs more brain power than I have left tonight.

I mocked up a rough byte level thing that only returns results bounded by the current edit window, but that needs to be expanded to return results for the biggest bounding blocks of either this or the previous pass.

[fmoralesc]

I as toying with something like this, which gets a list of the elements that overlap a byte:

def get_overlapping(source, index): 
         offsets = libvim_commonmark.get_offsets(source) 
         overlapping = {} 
         for i in offsets: 
             data = offsets[i] 
             if index >= data['start'] and index <= data['end']: 
                overlapping[i] = data 
        return overlapping 

Calculating the offsets is more or less cheap, so we can do this before anything else. Instead of the full data including groups, we could use only the byte positions.

[alerque]

Yes and no. That's exactly what we need ... and we can even punt it to Rust for an order of magnitude faster processing: get_offsets_in_byte_range() or whatever.

The trickier issue is that we need it before and after the buffer change.

[alerque]

@fmoralesc I think I have this working more or less like your get_overlapping() idea now so that we only return and process a list of tags that cover the byte range of changed lines.

It works for inline cases, but it doesn't update when block nesting is affected outside of the scope of the actual edited lines.

[alerque]

As of now this partial update system flips out if you delete lines and hence make the buffer shorter.

We're getting close to past the point where we need a test suite ;-)

[alerque]

This work suffers from an issue related to #22. The last-parsed ranges being used to detect affected blocks are saved globally. With more than one buffer open this produces updates across bogus ranges. Either the state needs to move to the Vim/Lua side or per-buffer range maps need to be saved in Rust.