Support LSP Semantic Tokens

[marwan-at-work]

Check for existing issues

• [X] Completed

Describe the feature

Semantic Tokens allow for extra helpful highlighting that the language server might have better context over.

For example, in Go many things are well highlighted via Semantic Tokens which are all explained here.

My personal favorite is being able to see formatting directives easily in strings:

which is not currently supported in zed:

Thanks!

If applicable, add mockups / screenshots to help present your vision of the feature

No response

[JosephTLyons]

These are different requests, but this one feels like its in a similar category:

• https://github.com/zed-industries/zed/issues/5345

[filiptibell]

I've been trying to port my custom One Dark theme to Zed, everything went pretty smoothly, except for more complex code. Unfortunately some sections end up being almost unreadable compared to VSCode, where semantic tokens are available. VSCode on the top, and Zed on the bottom:

note how customizations are missing for mutable, macros, and enum members.

Here's another example of mutable vs non mutable semantic token highlighting:

more useful than one might initially think since it applies to variables / parameters and method calls.

It would also be neat if semantic tokens in Zed supported rules the same way VSCode does using globs/stars ("*.mutable": "#aabbcc") but that's not as important.

[DotRed108]

bump

[michaelangeloio]

This is the only blocker for me

[7jrxt42BxFZo4iAnN4CX]

This really helps to navigate the code. We are waiting impatiently.

"editor.tokenColorCustomizations": {
"comments": "#9D9D9D",

"textMateRules": [
    {
        "name": "Rust function definition name",
        "scope": [
            "meta.function.definition.rust entity.name.function.rust",
        ],
        "settings": { "foreground": "#FF90F0" }
    },
    {
        "name": "Rust function definition params",
        "scope": [

            "meta.function.definition.rust variable.other.rust",
            "meta.function.definition.rust keyword.operator.key-value.rust",
            "meta.function.definition.rust punctuation.brackets.round.rust",
        ],
        "settings": { "foreground": "#F06BDD" }
    },
],
},

[DotRed108]

bump

[lylythechosenone]

Is anybody working on this? It is genuinely the only thing stopping me from daily driving zed.

[xpe]

Is anybody working on this? It is genuinely the only thing stopping me from daily driving zed.

First, I understand that people want their top bug squashed or feature implemented. But when this kind of level of intensity shows up in bug reports, it can be hard for many of us to take at face value. Is this really a deal-breaker issue? How much of it is exaggeration?

So, for those that feel strongly about this issue, please help me understand:

• why is this bug a high priority?
• what are the sub-components of the problem? (is this issue blocked by something else?)

(Caveat: I'm not with the Zed organization.)

[lylythechosenone]

Gladly.

why is this bug a high priority?

First, I wouldn't call it a bug, but a missing and much-needed feature. Regardless, semantic tokens are an important feature for a few reasons:

• Code is much, much easier to understand when things are highlighted with the correct color regardless of context. It's much easier to track where different types and values are referenced. Consider a Rust module where an enum variant called A is used. Reading the code in Zed, one would assume that A is actually a constant, and not an enum variant, since that is the highlighting Zed currently gives it.
• Semantic token customization allows people to shape the way code is presented to them, which is important for accessibility as well as for everyday use. In VSCode, I have unsafe function calls set to underline, and mutable calls set to bold. That helps me understand what the code is doing without checking the signature of every function.
• Semantic tokens are supported in nearly every major editor with LSP support. (Even Lapce has semantic token support, iinm.) Why not Zed?

what are the sub-components of the problem? (is this issue blocked by something else?)

I am not familiar with Zed's internals, but the language server provides semantic tokens (if supported). It would be a matter of replacing tree-sitter highlighting with semantic tokens whenever they are available. I don't think anything is blocking this.