Improve scalability

[fabiospampinato]

The amount of work this extension performs scales with the number of lines, basically a file with 1000 lines will take 10 times more to decorate than a file with 100 lines.

At the moment, when the document changes, we trash all the previous decorations and start over, this will cause lags if the file is big enough.

Most of the times only one line or few lines change, usually there's no need to refresh the decorations for all the other lines.

Possible improvements:

• [x] Maybe deprecate multi-line code blocks (wrapped in ```), they make many things harder, including performance optimizations.
• [ ] Cache untouched lines' decorations
• [ ] Cache projects' tokens
• [ ] Cache regexes results, right now "negative regexes" get run multiple times

[fabiospampinato]

As of v2.0.0 performance should be much better, I've been editing a 7k+ lines file without too many problems. Editing smaller files should feel pretty quick.

Some previous O(n) scenarios are now O(1):

• Editing an undecorated line
• Editing a line where the decorations are fixed
• Editing a line where the entire line is decorated

I've also optimized the code quite a bit, but unfortunately I couldn't properly cache decorations of unedited lines, turns out VSC's APIs are pretty lacking in this regard. I'm going to keep this open as performance could be improved even further, but there's not much that I can do without proper APIs, upstream issues:

• https://github.com/Microsoft/vscode/issues/50346
• https://github.com/Microsoft/vscode/issues/50415

[fabiospampinato]

Another currently O(n) scenario that could be optimized to be O(1) is this:

• Editing a file where decorations aren't fixed, but they are at the very end of the line. Basically editing a todo with one or more tags at the end.

[marco-m]

Microsoft/vscode#50346 has been closed as dup of Microsoft/vscode#585

[fabiospampinato]

It looks like there's a "Semantic tokens API", could be interesting: https://github.com/microsoft/vscode/issues/86415

[beeplin]

I came from https://github.com/fabiospampinato/vscode-todo-plus/issues/210 and is suffering from the same lag in a 100 lines todo file. Have to reload vscode window again and again to avoid the lag. :(

[beeplin]

I guess the lag is caused by redrawing the whole document each time the document is changed -- that is, each time a key is pressed. Maybe we could improved it by introducing some kind of debouncing behavior. When keep typing continuously, it can wait until the typing stoped for, say, 1 second.