Extensions - Language Features: Implement semantic highlighting

[bryphe]

The extension host provides some APIs for semantic highlighting:

    $registerDocumentSemanticTokensProvider(handle: number, selector: IDocumentFilterDto[], legend: modes.SemanticTokensLegend, eventHandle: number | undefined): void;
    $emitDocumentSemanticTokensEvent(eventHandle: number): void;
    $registerDocumentRangeSemanticTokensProvider(handle: number, selector: IDocumentFilterDto[], legend: modes.SemanticTokensLegend): void;

https://github.com/onivim/vscode-exthost/blob/0d6b39803352369daaa97a444ff76352d8452be2/src/vs/workbench/api/common/extHost.protocol.ts#L400

and

    $provideDocumentSemanticTokens(handle: number, resource: UriComponents, previousResultId: number, token: CancellationToken): Promise<VSBuffer | null>;
    $releaseDocumentSemanticTokens(handle: number, semanticColoringResultId: number): void;
    $provideDocumentRangeSemanticTokens(handle: number, resource: UriComponents, range: IRange, token: CancellationToken): Promise<VSBuffer | null>;

https://github.com/onivim/vscode-exthost/blob/0d6b39803352369daaa97a444ff76352d8452be2/src/vs/workbench/api/common/extHost.protocol.ts#L1508

We need to wire these APIs up, and integrate them into our language feature / syntax highlighting pipeline, to support languages that use these rich semantic highlights.

Some extensions to test with semantic token support:

• Rust: https://github.com/rust-analyzer/rust-analyzer
• Clang: https://open-vsx.org/extension/llvm-vs-code-extensions/vscode-clangd
• C#: https://github.com/OmniSharp/omnisharp-vscode

[lougreenwood]

I was just about to post an issue about this in relation to differences between the same theme in VSCode & Oni2.

But I noticed that switching off semantic highlighting in VSCode will render things (virtually?) identically to Oni2, so I guess the discrepancies are semantic highlighting... 😄