Syntax highlighting sometimes gets out of sync

[DanTup]

I had this issue raised at https://github.com/Dart-Code/Dart-Code/issues/5082 and although I'd only seen it a few times in the past, I just saw it again today. I was typing in a file, and a bunch of colours got out of sync in my editor.

While my first thought was that it was an issue with the semantic token or textmate info, the "Inspect Tokens and Scopes" command shows in the tooltip that the same scopes are applied to the whole of words that are coloured differently.

Unfortunately I don't have reliable steps to reproduce, it just happens sometimes (it might have started when I saved a file and briefly saw "VS Code is waiting for Code Actions").

Here you can see that VS Code says the foreground color is the same for two positions in this word, yet one part is pink and one part is blue:

https://github.com/microsoft/vscode/assets/1078012/d85d8f60-ebd1-4d56-bbf1-51c67af61390

[hediet]

This is concerning. Since when did you see this? The last big change in tokenization is a year ago.

Is it correct that this only happens with semantic colorization?

[DanTup]

@hediet I've seen it before and not too recently, so it may have been around for a while. I think I previously thought it was a Dart bug and because I couldn't repro I hadn't dug into it. It was only when it occured yesterday (shortly after reading a report from a user seeing the same) that it occurred to me to check the tokens like this. So my feeling is that it's probably been around a while.

I don't know if it's related, but just before I saw it, the Dart server responded slowly to a request for code actions which I had set for on-save, so I saw the popup like "Fetching code actions". It's possible I clicked cancel (since I was just trying to save the file and new no code actions would make changes). I recall also seeing some cancelled LSP requests in the log, though I assumed they were just from me moving around the files and could be unrelated.

Next time I see it, I'll try to take note of what happened (and review the logs) to see there might be any pattern. It certainly seems odd for the colours in the editor and this popup not to match, but I'm not familiar with how the implementation here works.

[DanTup]

@hediet I think I was able to reproduce this a few times by:

• typing quickly
• hitting Ctrl+Z to undo (probably before the last semantic tokens request has completed?)

I'm struggling to repro on demand though so those steps may have been coincidence.

I can't be certain that the Dart server isn't returning incorrect info (eg. something is getting out of order during async handling of changing the file and responding to semantic tokens), however I would that a) VS Code will cancel and re-issue a semantic token request after the last edit (so any inconsistency would be short-lived) and b) the popup shown in the screenshots above would be consistent with the colouring.

[DanTup]

I just noticed this again after changing Git branch. I changed git branch in GitHub desktop, which brought my working changes onto the new branch, then I switched back to VS Code and it looked like this. I can't be certain the issue occurred during the branch, but I think so.

[DanTup]

I feel like I'm seeing this much more today, but without any real pattern. The "Inspect Tokens and Scopes" popup still seems to show the full token text at the top which doesn't match what's being coloured.

I've just been doing normal editing and typing in the editor, nothing that stands out as unusual.

[hediet]

@DanTup you use the latest dart extension, right? I'll try these days if I can reproduce with that.

[DanTup]

Yep (actually the pre-release, though there's no real difference).

It's certainly possible there are bugs coming from Dart here, but since in the tooltips the colouring doesn't match the token text shown at the top (eg. in my latest screenshot it shows "VsCodeApi" in the hover, but that token is coloured in two different colours) I feel like something isn't right in VS Code even if Dart is doing something bad.

If I'm able to figure out a good way to repro and get the LSP logs I'll let you know. I did see it a few times yesterday and it seemed like I could repro by just deleting and re-typing a few characters, but after a few attempts it stopped happening and I wasn't able to get a log of it.

[Number-3434]

I've noticed this bug in TypeScript and JavaScript as well.

[DanTup]

Not sure if the same issue, but some complaints on Reddit about Python showing bad colouring too:

https://www.reddit.com/r/vscode/comments/1ega6qc/getting_real_sick_of_vscode_having_messed_up/

[peaceshi]

@DanTup do you use Intel integrated GPU? NVIDIA or AMD GPU has same issue?

[DanTup]

I'm using an dedicated AMD card (6950 XT with latest stable drivers).