Cannot highlight U+200B and unhighlight multiple languages at the same time

[freshchild]

Does this issue occur when all extensions are disabled?: Yes

• VS Code Version: 1.76.0
• OS Version: Windows 11 22H2

Steps to Reproduce:

1. Input U+200B​ and some unicode characters with multiple languages.

U+200B->​<-U+200B

- test
- 测试
- テスト
- тест

2. If both .invisibleCharacters and .nonBasicASCII are true, then

3. However, if .invisibleCharacters is true, and .nonBasicASCII is false, then

.code-workspace:

{
    "folders": [
        {
            "path": ".\\",
            "name": "Test"
        }
    ],
    "settings": {
        "editor.unicodeHighlight.includeStrings": true,
        "editor.unicodeHighlight.invisibleCharacters": true,
        "editor.unicodeHighlight.nonBasicASCII": true,
        "editor.unicodeHighlight.allowedLocales": {
            "_os": true,
            "_vscode": true,
            "zh_cn": true,
            "ja": true,
            "ja-jp": true,
            "ru": true,
            "ru-ru": true,
        },
        "editor.unicodeHighlight.ambiguousCharacters": false,
        "editor.unicodeHighlight.includeComments": true,
    }
}

It seems that there is no document related to editor.unicodeHighlight.allowedLocales standard input, so it is written according to the Visual Studio Code Language ID, If there is anything wrong, please correct it.

[hediet]

I cannot reproduce

[freshchild]

I cannot reproduce

Sorry, after retesting in 1.77.3, it seems to be reproduced only on .md:

• highlight.md: Same results as my test
• rename highlight.md to highlight.js or whatever: Same as yours @hediet

I disabled all extensions in this workspace, so it should be caused by the editor.

[freshchild]

BTW, highlighting doesn't seem to work when U+200B is surrounded by specific characters, like:

Maybe it should be on another issue, just so you know.