Classify tokens according to semantic-highlighting proposal

[hackwaly]

https://github.com/microsoft/vscode-languageserver-node/blob/324d1039fef2b9622a9784c5cc49d9f6dfc11b65/protocol/src/protocol.sematicTokens.proposed.ts

export enum SemanticTokenTypes {
    comment = 'comment',
    keyword = 'keyword',
    string = 'string',
    number = 'number',
    regexp = 'regexp',
    operator = 'operator',
    namespace = 'namespace',
    type = 'type',
    struct = 'struct',
    class = 'class',
    interface = 'interface',
    enum = 'enum',
    typeParameter = 'typeParameter',
    function = 'function',
    member = 'member',
    property = 'property',
    macro = 'macro',
    variable = 'variable',
    parameter = 'parameter',
    label = 'label'
}

[EvgeniyPeshkov]

Hello @hackwaly . I'm very sorry, but I didn't get what exactly you want me to do? Do you want to adopt this tokens in Syntax Highlighter replacing current ones? Or do you ask me how would I map these 2 sets if I were you?

[hackwaly]

I mean to adopt these tokens in Syntax Highlighter replacing current ones. And some day we can switch to use semantic highlighting feature of vscode to implement highlighting

[sandstrom]

There are more details on semantic highlighting in the wiki: https://github.com/microsoft/vscode/wiki/Semantic-Highlighting-Overview

However, it's fairly early days for this feature, but I agree with @hackwaly that it would a good to have a look at it.

[EvgeniyPeshkov]

Hello guys. It seems the time has come. As I can see, there is a new semantic token api (in fact I'm not really sure how new it is (= ). Looks like quite a thing, I was waiting for it for a long time. I believe it's much faster than decorations. Also it'll keep highlighting in sync with themes. So looks like many problems are to be resolved. I'm planning to adopt semantic token api, meaning almost completely rewrite the extension. I'm quite busy at the moment, but will make it ASAP.

[razzeee]

Are you planning to incorporate tree-sitters highlighting? https://tree-sitter.github.io/tree-sitter/syntax-highlighting There only seems to be a rust/c api