Keystroke lag with projects

[IAmFletcher]

VS Code: 1.36.1-1 Todo+: 4.13.0

Had the same issue as #53 on a fresh Manjaro Linux install in a 100 line file.

Looked at Process Explorer and was jumping to between 2% and 11% CPU on a single toggle box, and stayed between 8% to 12% when continuously toggling box.

With statistics disabled, I was jumping to between 1-2% on a single toggle box, and remained stable at 5% when continuously toggling box. There was also no more noticeable keystroke lag.

It seems to be related to projects specifically. 100 tasks on a new file was fine, but 16 projects and 84 tasks produced the issues listed above. It also doesn't seem to matter if they're nested, or not.

Tried on the latest Insiders (1563341490-1), and performance improved by about 2-3% whether statistics was enabled, or not. There was also less keystroke lag with statistics enabled, although it was still noticeable.

[fabiospampinato]

I'm closing this as a duplicate of #54.

Unfortunately we are pretty limited by the APIs provided by VSCode, writing a performant dynamic syntax highlighter like ours gets pretty complicated when dealing with those. If you have more ideas on how to help improve the situation I'm all ears.

[squidfunk]

I'm having the same problem. Specifically the more edits I make to the TODO file after I open VS Code, the slower and slower it gets. Adding a simple space will make the CPU jump to 70-80%. My current solution is to close VS Code and reopen it, but I've been doing this for months now and is really annoying.

[fabiospampinato]

@squidfunk Wait does reloading the window fix the problem for you? If so we might have some kind of memory leak or something. It also also be something happening at the vscode-level that we have no control over.

[squidfunk]

Here's a gif showing the severe lag in a file with ~100 tasks and ~20 projects. Reloading does not solve the problem.

[squidfunk]

Here's another recording that hints that syntax highlighting is probably triggered on every keystroke and chained together. It takes a long time until the highlighter catches up.

EDIT: If I then close VS Code and open it again, syntax highlighting still lags, but not as severe as before. Same file, same amount of tasks.

[squidfunk]

I tested the projects regex from here on regex101.com with my file – it had more than 10k steps. However, the projects regex is only a singular regex, so I guess the sum of regexes makes it feel very, very slow.

https://github.com/fabiospampinato/vscode-todo-plus/blob/ad223abe4c48fbf5eef02d3dce75fd1bd2d399f6/src/consts.ts#L42-L71

I played a little around with the regex and could cut step count in half by just replacing the (.+:) in the middle with ([^:\n]+:), as the former will lead to excessive backtracking. When I cut down the regex to ^[^\S\n]*([^:\n]+:), which also has some false positives like tasks with colons in them, I get the step count down to 3k steps. In my opinion this shows that the regex approach does not scale well.

On a side note: I've just implemented a CSS parser and first used a lot of regular expressions. It was insanely slow and also wrong, as CSS is not a regular grammar. Then, following a tokenization-based approach as described by the official specs, I managed to improve performance by a factor of 20-40, depending on the input. Parsing time now also scales linearly with input size. I would recommend refactoring the regular expressions by using a parser generator. My testing has shown that Chevrotain is very fast and easy to work with.

[fabiospampinato]

@squidfunk

• Can you check if you can still reproduce the issue in VSCode Insiders with only Todo+ installed?
• If you can, can you send me a file that reproduces the problem?
• I see from your screencasts that your tasks are being rendered in italic, that doesn't look right to me, what's up with that?
• I agree that a real parser/tokenizer would deal better performances, but here the biggest issue is that we are re-decorating the entire thing most of the times, VSCode's APIs are pretty limiting in this regard. Perhaps there's a way to cache decorations for untouched lines though, I haven't explored this area yet.

[squidfunk]

• Can you check if you can still reproduce the issue in VSCode Insiders with only Todo+ installed?

Will need to install insiders first, but it's probably related to the extension itself.

• If you can, can you send me a file that reproduces the problem?

I replaced all characters with X in my original file, so it should reproduce the issue:

# XXXXXXXXX (X) XXXX-XXXX XXXXXX XXXXXX <XXXXXX.XXXXXX@XXXXXXXXX.XXX>

# XXX XXXXXX XXXXXXXX. XX XXXX XX XXXX XXXXXXXX XXXXXXX(X) XXX XX XXXX,
# XXXXXXXXXX, XXXXXX XX XXX XXXXXXXXX XXXXXX, XX XXXXXXXXXXX, XX XXX XXXX XX
# XX XXX XXXXX, XXXXXXXXXX, XXXXXXXXXX, XXXXXXXXXXXX, XXXXXXXXX, XX XXXXXXXXX
# XXXXXXX XXXXX XXXXXXX XXXXXXXXXX.

# XXX XXXXX XXXXXXXXX XXXXXX XXX XXXX XXXXXXXXXX XXXXXX XXXXX XX XXXXXXXX XX
# XXX XXXXXX XX XXXXXXXXXXX XXXXXXXX XX XXX XXXXXXXX.

# XXX XXXXXXXX XX XXXXXXXX "XX XX", XXXXXXX XXXXXXXX XX XXX XXXX, XXXXXXX XX
# XXXXXXX, XXXXXXXXX XXX XXX XXXXXXX XX XXX XXXXXXXXXX XX XXXXXXXXXXXXXXX,
# XXXXXXX XXX X XXXXXXXXXX XXXXXXX XXX XXX-XXXXXXXXXXXX. XX XX XXXXX XXXXX XXX
# XXXXXXX XX XXXXXXXXX XXXXXXX XX XXXXXX XXX XXX XXXXX, XXXXXXX XX XXXXX
# XXXXXXXXX, XXXXXXX XX XX XXXXXX XX XXXXXXXX, XXXX XX XXXXXXXXX, XXXXXXX
# XXXX, XXX XX XX XX XXXXXXXXXX XXXX XXX XXXXXXXX XX XXX XXX XX XXXXX XXXXXXXX
# XX XXX XXXXXXXX.

XXXXXXX:

  [XXXXXXX, XX.XX]
  ✔ XXX XXXXXX XXXXXXXXXX @XX @XXXX(XX-XX-XX XX:XX)
  ☐ XXXXXXXXX XXXXXXX XXXXXXXXX: XXXXXXXXX XXXXXXX @XX
    – XXXXXXXXX `XXXXXXXXXXX` XXXXXXXXXXX XXXX
  ☐ XXXXXXXXX XXXXXXX XXXXXX: XXXX XXXXXXX XX XXXXXXXX @XX
    - XXXXXXXXX XXXXXXXX XXXXXXXX XXXXXXX XXXXXX
    - XXXXXXXXX XXXXX XXXXXXXXXXX / XXXXXXXXXX
    - XXXXXXXXX XXXXXX XXXXXX (XXXXXXXXXXXX, XXXXX)

  [XXXXXXXXX, XX.XX]
  ☐ XXXXXXXX XXXXXXX XXXXXXXXX XXX XXXXXX @XX
  ☐ XXXXXXXX `XXXXXXXXXXXXXXXXXXX` @XX
  ☐ XXXXXXXX XXX-XXXXX XXXXXXX @XX
  - `XXXXXXXXXX`
  - `XXXXXXXXXXXXXXXXXX`
  - `XXXXXXXXXXXXX`

  [XXXXXXXX, XX.XX]
  ☐ XXXXXXXX XXXXXX XXXXXXXXXX @XX
    - XXXXXXXXX X XXXX XX XXXXXXXXXX XX XXXXXX
    - XXX XXXXX XX XXXXXXXX: XXXX XX XXXX
    - XXX XXXXXX XXXXXX, X.X. `XXXXXXXXXX(-XX-XXXXXXXXX-XXXXXXX-XXXXX)`
    - XXXXXXXXX XXXXXXX XX XXXXXXXX XXXXXXX
    - XXXXXXXXX XXXXXXXXX XXXXXXX XX XX XXXXXXXXXXXX

  [XXXXXX, XX.XX]
  ☐ XXX XXXXXXXX XXX XXXXXXX XXXXX XXX XXX @XX

  [XXXX XXXX]
  ☐ XXX XXXXXXXX XX XXXXXX XXX XXXXXXXXXXX @XX
    - `XXXXXXXXXX`
    - `XXXX`
  ☐ XXXXXXXXX XXXXX XXX XXX-XXXXX XXXXXXX @XX
  ☐ XXXXXXXXX XXXXXXXX XXXXXXX XXXXXXX @XX
    - `XXXXXXXXXXXXXXXXX` (XXXXXXX X,X)
  ☐ XXXXXXXXX XXXXXXXXX XXX XXXXXXXX XXXXX XXXXXXXXXX @XX
  ☐ XXXXXXXX XXXXX XXXXXXXXXX XX XXXXXX XXXXX XXXXXXXX @XX
    - X.X. `<XXXXX>` -> `[XXXXXXXXXXXX, XXXXXXXXXXXX<"/">, XXXXXXXXXXXX]`
    - X.X. `XXXXXXXXXXXXX` ...
  ☐ XXXXXXXXX XXXXXXXX XXXXXXXXXXX XXXXX @XX
  ☐ XXXXXXXXX XXXXXXXX XXXXXXXXXXX XXXXXXXXXX @XX
  ☐ XXXXXXX `XXX/XXXX` XX X.X.X @XX
    - XXXXXX XX `XXXXXXXX`
  ☐ XXXXXXXXX XXXXXXXXXXX XXXXX @XX
    - XXXXXXXXX XXX-XXXXXXX XXXXXXX (XXXX XXXXXXXX XXXXX)
    - XXXXXXXXX XXXXXXXXXX XXX XXXXXXXXXX XX XXXX XXXXX
    - XXXXXXXXX XXXX XXXXXXXX XXX XXXXXXXXXX XXX XXXXXX
  ☐ XXXXXXX XXXX XXXXX + XXXXX @XX
    - `XXXXXXXXXXXXXXX`, XXX...
  ☐ XXXXXXXXX XXXXX XXXXXXXXXX
    - XXXXX://XXXXXX.XXXXX.XXX/XXX-XXXXXXX-X/#XXXXXX-XXXXXXXX
    - XXXXXXXXX XXXXXXXX XXXXX, X.X. `XXX%` (XXXXXXXX)
    - XXXXXXXXX XXXXXXXX XXXXX, X.X. `XXX%` (XXXXXXX XX XXXXXXX)
    - XXXXXXXXX XXXXXXXXX XXXXX, X.X. `XXX%` XX. `XXXX` (XXXXXXXX XX. XXXXXXX)
    - XXXXXXXXX XXXXXXXX XXXXX, X.X. `XXXX` -> `XXXX` (XXXXXXXXXX)
    - XXXXXXXXX XXXX XXXXX, X.X. `XXXX` -> `XXXXX`
    - XXXXXXXXX XXXXXX XXXXX, X.X. `X.XXX` -> `XXX`

XXXX:
  ☐ XXXXXXXX XXXXXX XXXXXXXX XX `XXXXXXXXXXXXXXXXXXX`
    - "XXXXXXXX 'X' XX 'X' XX ..."
  ☐ XXXXXXXXX XXXXXXX XX XXXXXX XX `XXXXXXXXXXX.XX`
  ☐ XXXXXXXXX XXXXXX `XXX | XX` XXXXXXX-XXXXX XXXXXX!
  ☐ XXXXXXXXX XXXXXX-XXXXX XXXX-XXXXXXX XXXXXX, X.X. XXX `@XXXXXXXX`
  ☐ XXXXXXXXX XXXXXXX XX XXXXXXX XXXXX
  ☐ XXXXXXXXX XXXXXXXX XX XXXXXXXX XXXXXXXX XXXXXX, X.X. `XXXXXXX-XXX: XX`
    - XXXXX://XXXXXXXXXXXXX.XXX/X/XXXXXXXX/XXXXXXX
  ☐ XXXXXXXXX XXXXXXXXX XXXXXX-XXXXXX
    - XXXXX://XXXXXXXXX.XXXXXXX.XXX/XX-XX/XXXX/XXX/XXX/XXXXXX-XXXXX
  ☐ XXXXXXXXX XXXXXXX XX XXXXXXXXXX XX XXXXXXXXXX, X.X. `XXXXXXX`
    - XXXXX://XXX.XX.XXX/XX/XXXXX/XXXXX/XXXXXXXXXX.XXXX
  ☐ XXXXXXXXX `<-XX-XXXXXX>`, `<-XX-XXXXX>`, ...
    - ` XXXX -X "XXXXXX:XXXXXXXXXXXXXXXX" XXXXX/XXXXXX/XXXX/XXXXX/*.XXX`
  ☐ XXXXXXXXX XXXXXXXXX XXXXXXXXX
  ☐ XXXXXXXX XXXXXXXXX XXXX (X-XXXX)
  ☐ XXXXX XX-XXXXXXX XXXXX

XXXXXX:
  ☐ XXXXXXXXXX XXXXXXX XXXX XXXXXXXXXXXX (XXXXXXXXXX, XXXXXXXX, XXXXX)
  ☐ XXXXXXX XXX
    - XXXX XXX XXXXX XXXX XXXXX XXX XX XXX XXX XXXXXX
      - XXXXX://XXXX.XXXXXX.XXX/XXXXXX/XXXXXXX
  ☐ XXXXXXXXXXX (XXXXXXXXXXXXXXXX, XXXXX, XXX...)
  ☐ XXX XXXXX XXX XXXXXXX (XXXXXXXX)
  ☐ XXX XXXXX XXX XXXXXX (XXXXXXXX)
  ☐ XXXXXX XXXXXXXXX XXX XX XXXXX
  ☐ XXX XXXXX-XXXXX XXXXXXXXX
  ☐ XXXXXXXX XXXXXXXXX / XXXXXXXX

XXXXXXXXX XX XXXXXX XXXX XXXXXXX XX XXXX:
  - XXXXX XXX XXXXXXX XXXXXXXX XXXXX XX XXXXXX XXXXX XXX / XX XXX, XXX...
  - XXXXX XXXXX XXX XXXX XXXXXXXX? (XXX XXXX XX XX XXXXXXX XXXXXXXX)
  - XXXXX XXXXX XXX XXXXXXXXXXX XX XXXX XXXXXXXX? XX XXXX XXXXXX XXXXXX?
  - XXXXXXX -> XXX XXXXXXXXXX XXXXX (XXXXXXXXX XXXXXXXXXX)
    - XXXXXXXXXX XX XXXXXXXXXX XX XXX XXXX XXXXXXX - XXX XXXX, XX XXX XXXXXXX XXXXXXXXXX
    - XXXXXXXXXXXX XX XXXXX XXXXXXXX XXX XX XXX, XXXXXXX XXXX XXX XXX XX XXXXXXX

---

XXX:
  ☐ XXXXX XX XXXXXX: `XXXXXXXXXX("(XXXX: -X.XXXX)")`
  ☐ XXXX XXXX XXXX XXXXXX `<XXXXXX>` XXX. XXXX XXXXX XXX XXXXXXX
    - XXX?

XXXX:
  XXXXXX:
    XXXXX XXXX:
      ☐ XXX XXXXXX XXXXXXXXX XX XXXXXXXXX
        - XXXXXX XX XXXXXX XXXXXX XXXXXXX XXXXXX!!!
      ☐ XXXXXXX XXXXXX XXXXXX XXX. XX XXXXX
  XXXXXX:
    ☐ `XXXXXX.XXXXXX` = XXX XXXXXXX XX XXXXXXX X XXXXXX XX X XXX
    - XXXXXXX XXXXXXX XXX XXXX, XXXXXXXXX XXXX XXXXXXXXXX XXX XXXXXXXXX XXXXXX/XXXX XXX XXXXXXX

  XXXXXXXXXX:
    XXXXXXX XX XXXXXXX:
      ☐ XXXXXXXX XXXXXXXXXXXXX XXXXX XX XXXXXX XXXXXX

  ☐ XXX XXXXXX XXXX XX XXXXXX, XXXXXXX XX XXXXXXXXX XXX XXXXXXXXX XX XXXXX
  ☐ XXX: XXXX-XXXXXXX XXXXXXXX XX XXXX XXXXXXX XX XXXXXXXX

XXXX:
  ☐ XXXXXXX XXXXX XXXXXXX, XXXXXXXX XXXXXXX XXXXX
  ☐ XXXXXXXXXXX `XXXXXXXX/XXXXX`
  ☐ XXXXX XXXXX XXX XXX XXXXXXXX XXXXXXXXXXXX
  ☐ `XXXXXX` => `XXXXXXX` / `XXXXXXX` ...?
  ☐ XXXX: XXX XXXX XXXX XXXXXXXXXX!!!
  ☐ XXXXXXXX XXXXX XXXX XX XXXXXXXXXX... XXXXXXXXX XXX XXXX XXX'X XXXX XXX
  ☐ XXXXX/XXXXX XXXXXXXXXXXXXXX
    XXXXXXXXX XXX. XXXX XXXXXXX. XXXXXXX, XXXXX XXXXX XXXX XX XXXXXXXXXX XXXXX
    XXX XXXXXX XXXX --> XXXX XXXXXXXXXX... XX XXXXX XXXX XXXX XXXXX XXXXXXXXXX!
  ☐ XXXXX XXX `XXXXXXX: XXXX`
  ☐ XXXXX XXX `XXXXXXX: XXXXXXXX`

  // XX: XXXXXXX XXXXX XX X (XXXXXXXX XX XXXX XXXX XX XXXXXXX!)
  // XX: X-XXXXXX - XXXXX XXX XXXXX XXXXXX XXXX XXXX

  XXXXXX:
    XXXXXXX:
      ☐ XXX XXX XXXXXXX XXXXXXXXXXXX
      ☐ XXX XX XXXX XXXXXX XXXXXXXXX
      ☐ XXXXX X XXXX XXXXXXX XXXXXXXX XXX XXXXXXXX
    XXXXXXXXXXX:
      ☐ XXXXXXXXX `XXXX-XX`-XXXXX XXXXXXXXXXX
      ☐ XXXXXXXXX XXXXXXX XXX XXXXXXXXXXX XXXXXX
        - X: XX XXXXXXXXXXX
        - X: `XXXX-XX` XXXXXXXXXXX
    XXXXXXX:
      ☐ XXXXX XXXXX XXX XXXXXXXX (XXXXXXXX, XXX.)
      ☐ XXXXX XXXXX XXX XXXXXXX
      ☐ XXXXX XXXXX XXX XXXXXXXXXXX

  XXXXXX:
    ☐ XXXXX XX XXXXXX
    ☐ XXXXX XX XXX XXXXXXXX
    ☐ `>>>` XX `>>`
    ☐ XXXXXXXXX XXXXX `XXXXX` XXXX XXXXXXX XXXXXXX XXXXXX
    ☐ XXXXXX `XXXXXXXXXXXXX` XXXX `XXXXXXX.XXXXXX.XX`

  XXXXX:
    XXXXXXXXX:
      ☐ XXXXX XXXXXXXXX XXX XXXXXXXXXXX (`XXXXXXX` XXX `XXXXXXXXX`) @XX
      XXXX XXXX XXXXXXXX:
        XXXXXXX:
          XXXXX-XXXXXX:
            ☐ XXXXX XXXXXXXXXXX XXXX XXXXXXXXXX (XXXXXXX XXXXXXXXXX)
            ☐ XXXXXXXX XXXX XX XXXXXXX XXXX-XXXXX XXX XXXX XXXXXXXXXX (XXXXXXX-XXXXXX)
          XXXXXXXX-XXXXXX:
            ☐ XXXXXXXX XXXX XXXXXX/XXXXXXX
            ☐ XXXXXXXXX XXXXXXXXX
            ☐ XXXXXXX XXXXX
        XXXXXXXX:
          XXXXXX:
            ☐ XXXXX XXXX XXXX XXXXXXXXXXX
            ☐ XXXXXX XXX XXXXX XX XXXXXXXX (XXXXXX, XXXXXX XXXXXX, XXX.)
            ☐ XXXXXXXXX XXXX XX XXXXXXXX XXXXXX XXXX XX XXXXXX XXXXXXXX XX XXXXX
            XXXXXXXXXX XXXXXXXX:
              ☐ XXXXXXXXXX?
              ☐ XXX XX XXXXXXX XXXXX XXXXXX XX XXXXXX XX XXXX XXXXX (XXXX?)
          XXXXXXX:
            ☐ XXXXX XXXXXX-XXXXX XXXXXXXX (XXXXXXXXX XXXXXXXX)
            ☐ XXXXXXXX XXXXXXX XXXXXXXXXX
    XXXXX:
      XXXXX XXXXXXX:
        XXXXXXXX:
          ☐ XXXXXXXXX XXXXX XXXXX XXXXXXXX
          ☐ XXXX XXXXX XXX XXXXX XXXXX XXXXXXXX
      XXXXXXXX XXXXXXX:
        ☐ XXXXXXXXX XXXXXXXX XXXXXXXX
          ☐ `XXXXXXXXXXXXXXX/XXXXXXX = XXXXXXXX`
          ☐ XXXX XXXX XX XXXXXX XXXX XXX `:XXXXX` XX XXX XXXXXXXXX
        ☐ XXXXXXXXX XXXXXX-XXXXX XXXXXXXXX
        ☐ XXXXX XXXXX
          - `XXXXX/XXXXX/XXXXXXXX/XXXXXX/XXXXXXXXX`
          - `XXXXX/XXXXX/XXXXXXXX/XXXXXX/XXXXXX-XXXXX`
          - `XXXXX/XXXXX/XXXXXXXX/XXXXXX/XXXXXX-XXXXXXX`
          - `XXXXX/XXXXX/XXXXXXXX/XXXXXX/XXX`
          - `XXXXX/XXXXX/XXXXXXXX/XXXXXX/XXXXXXXXX`
        ☐ XXXXXXXXX XXXXXXXX XXXXXXXX
          -> XX XXXX, XXX XXXXXXXX XX XXXXXXXXXX XX X XXXXXXXXXXX XX XXX XXXX
            XXXXXXXXX XXX <XXXX> XXXXXXX, XXX XXXXXXXX XX XXXXXXXXXXX XXXX XXX
            XXXXXXXX (XXXX XX XXXX XXXXXXX).
        ☐ XXXXX XXXXXXXXXX, XXXX XXXXXXXXX XXX. XX XXXXXXXXXXX, XXXXXXXX XXX XXXX
        ☐ XXXXX XXXXXXXX XXXXXXXXX XXX XXXXXXX XXXXXXXXXXX XXXXXX XXXXXXX
          -> XXXXX://XXXXXX.XXXXX.XXX/XXXXXXXXX/#XXXXXXXX
      XXXXXXXX:
        ☐ XXXXXXXXX XXXXXXXX XX `<XXXX>`, `<XXXXXX>`, XXX...
        ☐ XXX XXX XXXXX (XXXXX://XXXXXX.XXX/XXXXXXXXXXX/XXX-XXX-XXXXX) XXX XXXXXXXX
        ☐ XXXXXXX XXXXXX XX XXXXXX (XXXX XXXXXXX XXXXXX XXXX XXX XXX XXX XXXX)
        ☐ XXXXXXXXX XXXXXXX XXXXXXXX XXXXXXX
    XXXXXXXX:
      ☐ XXXXXX `XXXXXXXX` XXXX (XXXXXXX XXXX, XXXXXX, XXX...)
      XXXX:
        ☐ XXXXX XX XXX XX XXXXXXX XXXXXXXXX XX XXXXX XXX.
        ☐ XXXXX X XXX XXXXXXXX XXXXXXXXXX XXXXXX
      XXXXXXXXX:
        ☐ XXX XXXXXXXXXX XXXXXXXXXXX XX XXXX
    XXXXX/XXXXXX:
      XXXXXXX:
        ☐ XXXXX `XXXXXXX-XXXX` XXXX XXXX
          ☐ XXXXX XXXXX XXXXXXXX
          ☐ XXXXXXXXX
          ☐ XXXXXXXXXX
      XXXXX:
        ☐ XXXXX XXXXX
        ☐ XXXXX XXXXXXXX
        ☐ XXXXXXXXXX
        ☐ XXXXXXXXX
        ☐ XXXXXX
        ☐ XXXXX

  # XXXX-*
  XXXXXXX:
    ☐ XXX XX XXXXXXXXXX (XXXXX XXX XXXXXXX XXXXX!)
    ☐ XXXXXX XXX `XXXXXX: XXXXX` XXXXX XXX XXXXX XXXXXXX
    ☐ XXXXXXX XXXXXXXXX
      ☐ XXXXXXXX XXX XXXXXXXX XXXXXXX
        ☐ XXXXXX: XXXXXXX XXXXX XX XXXX XXXX XXXXX
        ☐ XXXX: XXXXXXX XXXXX XX XXXXXXX XX XXXX-XXXXXX XXX XXXX XXXXX
        ☐ XXXXXXX:
          ☐ XXXX: X XXX XXXXXXX XXX XXX
          ☐ XXXX: ? XXXXX://XXX.XXXXXXX.XXX/XXXXX
      ☐ XXX XXXXXXXXXXXXXX XXXXXX XXXXXXX
      ☐ XXXXXXXX - XXXXX XXXXXXXX (XXX/XX)

• I see from your screencasts that your tasks are being rendered in italic, that doesn't look right to me, what's up with that?

This has nothing to do with the extension or problem, it's just my font preference.

• I agree that a real parser/tokenizer would deal better performances, but here the biggest issue is that we are re-decorating the entire thing most of the times, VSCode's APIs are pretty limiting in this regard. Perhaps there's a way to cache decorations for untouched lines though, I haven't explored this area yet.

It would probably be a great performance improvement only redecorating the changed line. The tokenization approach however will also be fast enough in case of permanent redecoration.

[fabiospampinato]

The tokenization approach however will also be fast enough in case of permanent redecoration.

I'm not sure that's actually the bottleneck here, I've been editing a 10k lines .todo file without too much of a struggle, and that's surely an unrealistic file size, your ~100 lines files should be handled just fine, there's probably another issue here besides regexes.

[fabiospampinato]

This reproduces it for me:

I can't feel any junkiness editing that file. Try using the extension in a fresh VSCode installation, perhaps there's a conflict with some other extension.

[squidfunk]

I just tested with a fresh VS Code insiders installation. The only thing installed is Todo+:

https://imgur.com/a/Fw3sVpB

Exactly the same result.

[fabiospampinato]

I'm reopening as it seems clear that there's an addressable issue here, but unfortunately I'm unable to reproduce it.

You will have to try to debug this yourself I'm afraid.

[squidfunk]

You will have to try to debug this yourself I'm afraid.

Any directions on it? Happy to help. Is there an easy way to profile the highlighting, ideally to create a flame graph?

[fabiospampinato]

Read this, then run Debug Launcher: Auto.

A flame graph would be nice, unfortunately by debugging the extension VSCode's debugger is being attached, which doesn't provide the ability to record a flame graph as far as I know.

I'd say check how long the functions here and here take. And check how many times the decorations are being updated, if that's more than once per keystroke we might have a problem.

[squidfunk]

Found it. Here is a fresh CPU profile:

CPU-20190723T151847.077Z.cpuprofile.zip

[fabiospampinato]

Oh nice, I didn't even know that was possible.

[fabiospampinato]

Is all that taking place for a single keypress?

[squidfunk]

No, that is basically the same thing I did for the latest imgur video, i.e. extending a code block with further characters.

[fabiospampinato]

It'd be useful if you can export that (right click on the profile -> save...) so that I can import that on my end. Or I guess I'll set this up myself, but I've been pretty busy with other projects of mine to dedicate a ton of time to this.

[squidfunk]

I already attached the profile in the zip file with the screenshot. Here it is again: CPU-20190723T151847.077Z.cpuprofile.zip

[squidfunk]

Oh another hint: when I just open the TODO file with file type Markdown, everything works perfect - no lag, no delay, everything is updated instantly.

[fabiospampinato]

I already attached the profile in the zip file with the screenshot.

Oh right, sorry!

I can totally reproduce this by doing the same thing you're doing in the screencast, previously I was just editing todos/projects or writing quickly.

We have an optimization where if a decoration between edits is supposed to change, say the length of the text to decorate changes, but in both cases the decoration encompasses the entire line we just do nothing. It seems that this isn't actually working for your use case for some reason.

From the flame chart you provided I see lots of small things that could perhaps be optimized.

Ultimately though the definitive performance optimization would be to make the re-decoration process a O(number of edited lines) process rather than the currently O(number of lines) complexity in the general case. That's tricky to do because of VSCode's APIs, but there might be a way to do it.

Oh another hint: when I just open the TODO file with file type Markdown, everything works perfect - no lag, no delay, everything is updated instantly.

Yeah but then you miss on Todo+'s syntax highlighting.

The reason why Markdown's syntax highlighting is much more performant is because they are actually properly tokenizing things, via a .tmLanguage file. We can't provide a .tmLanguage file because some of our decorations depend on your configuration settings, so we can't just create a static .tmLanguage file.

[fabiospampinato]

I'll probably be able to take a closer look at this during one of the following weekends. If you'd like to optimize things yourself though PRs are more than welcome 👍

[squidfunk]

@fabiospampinato the reproduction (= hammering on the keyboard) was just to demonstrate the issue. The lagging mainly happens when I edit lines or move blocks of TODOs around, especially when I extend code blocks.

Ultimately though the definitive performance optimization would be to make the re-decoration process a O(number of edited lines) process rather than the currently O(number of lines) complexity in the general case. That's tricky to do because of VSCode's APIs, but there might be a way to do it.

Fully agree.

I'm an open source maintainer myself, so I know that time is a scarce resource. If you manage to find some time, it would be really great, but I understand that it might take a little. Nevertheless, I have to say: I love this extension. Thank you for creating and maintaining it!

[fabiospampinato]

Closing in favor of #54

[squidfunk]

The problem was fixed in #367 and the fix was released in v4.19.1 🥳