vscode-textmate-languageservice
🎉 This package has been adopted by the
vsce-toolroom
GitHub collective.This package is in maintenance mode & the Textmate technology is superseded by
vscode-anycode
, a quicker language service which leverages thetree-sitter
symbolic-expression parser technology.
Language service providers & APIs driven entirely by your Textmate grammar and one configuration file.
In order to generate language providers from this module, the Textmate grammar must include the following features:
multiple
and single
begin
and end
scopesnpm install vscode-textmate-languageservice
Browser support:
crypto
as a external (commonjs crypto
one in webpack).
This allows the library to avoid polyfilling the node:crypto
module.Advisory:
This package is stable with browser compatibility (
1.1.0
). But I recommend you watch out fortree-sitter
native integration intovscode
(issue). Maintainable & with faster retokenization, it is a Holy Grail ...Whereas this package depends on a well-written Textmate grammar and is a band aid of sorts.
If there is native
vscode
support for the language, find a Tree-sitter syntax online then suggest it in an Anycode issue. Otherwise, please open an issue on the community-maintained Treesitter syntax highlighter extension and someone might deal with it.
contributes
in the extension manifest (or textmate-languageservice-contributes
).vscode-textmate
(which can load PList XML, JSON or YAML grammars)../textmate-configuration.json
. You can also use textmate-languageservices
property of package.json
to map language ID to relative path.Example language extension manifest - ./package.json
:
{
"name": "lua",
"displayName": "Textmate language service for Lua",
"description": "Lua enhanced support for Visual Studio Code",
"version": "0.0.1",
"publisher": "",
"license": "",
"engines": {
"vscode": "^1.51.1"
},
"categories": [
"Programming Languages"
],
"contributes": {
"languages": [{
"id": "lua",
"aliases": ["Lua"],
"extensions": [".lua", ".moon", ".luau"],
"configuration": "./language-configuration.json"
}],
"grammars": [{
"language": "lua",
"scopeName": "source.lua",
"path": "./syntaxes/Lua.tmLanguage.json"
}]
}
}
Create a JSON file named textmate-configuration.json
in the extension directory. The file accepts comments and trailing commas.
If you only want to use the document and/or tokenization services, this file can be as simple as {}
.
Textmate configuration fields:
assignment
- optional (object
)separator
: Token to separate multiple assignments (string
)single
: Token to match single variable assignment. (string
)multiple
: Token to match multiple variable assignment. (string
)declarations
- optional (array
)dedentation
- optional (array
)ELSE
, ELSEIF
).indentation
with the decrementing value -1
.exclude
(string
)
VS Code glob pattern for files to exclude from workspace symbol search.indentation
- optional (object
)punctuation
- optional (object
)continuation
: Token scope selector for line continuation (to use in region matching). (string
)markers
- optional (object
)start
: Escaped regular expression for start region marker. (string
)end
: Escaped regular expression for end region marker. (string
)
Properties:symbols
- optional (object
)vscode.SymbolKind
value).Template for textmate-configuration.json
file:
{
"assignment": {
"single": "",
"multiple": "",
"separator": ""
},
"declarations": [],
"dedentation": [
"keyword.control.elseif.custom",
"keyword.control.else.custom"
],
"exclude": "{.modules,.includes}/**",
"indentation": {
"punctuation.definition.comment.begin.custom": 1,
"punctuation.definition.comment.end.custom": -1,
"keyword.control.begin.custom": 1,
"keyword.control.end.custom": -1
},
"punctuation": {
"continuation": "punctuation.separator.continuation.line.custom"
},
"markers": {
"start": "^\\s*#?region\\b",
"end": "^\\s*#?end\\s?region\\b"
},
"symbols": {
"keyword.control.custom": 2,
"entity.name.function.custom": 11
}
}
An example configuration file that targets Lua:
{
"assignment": {
"single": "meta.assignment.variable.single.lua",
"multiple": "meta.assignment.variable.group.lua",
"separator": "punctuation.separator.comma.lua"
},
"declarations": [
"meta.declaration.lua entity.name",
"meta.assignment.definition.lua entity.name"
],
"dedentation": [
"keyword.control.elseif.lua",
"keyword.control.else.lua"
],
"exclude": "{.luarocks,lua_modules}/**",
"indentation": {
"punctuation.definition.comment.begin.lua": 1,
"punctuation.definition.comment.end.lua": -1,
"keyword.control.begin.lua": 1,
"keyword.control.end.lua": -1
},
"markers": {
"start": "^\\s*#?region\\b",
"end": "^\\s*#?end\\s?region\\b"
},
"symbols": {
"keyword.control.lua": 2,
"entity.name.function.lua": 11
}
}
Extension code sample - ./src/extension.ts
:
import TextmateLanguageService from 'vscode-textmate-languageservice';
export async function activate(context: vscode.ExtensionContext) {
const selector: vscode.DocumentSelector = 'lua';
const textmateService = new TextmateLanguageService(selector, context);
const foldingRangeProvider = await textmateService.createFoldingRangeProvider();
const documentSymbolProvider = await textmateService.createDocumentSymbolProvider();
const workspaceSymbolProvider = await textmateService.createWorkspaceSymbolProvider();
const definitionProvider = await textmateService.createDefinitionProvider();
context.subscriptions.push(vscode.languages.registerDocumentSymbolProvider(selector, documentSymbolProvider));
context.subscriptions.push(vscode.languages.registerFoldingRangeProvider(selector, foldingRangeProvider));
context.subscriptions.push(vscode.languages.registerWorkspaceSymbolProvider(workspaceSymbolProvider));
context.subscriptions.push(vscode.languages.registerDefinitionProvider(selector, peekDefinitionProvider));
};
Extension code sample - ./src/extension.ts
:
import TextmateLanguageService from 'vscode-textmate-languageservice';
export async function activate(context: vscode.ExtensionContext) {
const selector: vscode.DocumentSelector = 'custom';
const textmateService = new TextmateLanguageService('custom', context);
const textmateTokenService = await textmateService.initTokenService();
const textDocument = vscode.window.activeTextEditor!.document;
const tokens = textmateTokenService.fetch(textDocument);
};
NB: If you would like to:
TextmateLanguageService
provider configuration..You can use the custom "textmate-languageservice-contributes"
property in package.json
:
{
"textmate-languageservice-contributes": {
"languages": [{
"id": "typescript",
"aliases": ["TypeScript"],
"extensions": [".ts", ".tsx", ".cts", ".mts"]
}],
"grammars": [{
"language": "typescript",
"scopeName": "source.ts",
"path": "./syntaxes/TypeScript.tmLanguage.json"
}]
}
}
Usage (example is for getting the token at the current cursor position):
const { getScopeInformationAtPosition } = TextmateLanguageService.api;
const editor = vscode.window.activeTextEditor;
const document = editor.document;
const position = editor.selection.active;
const token = await getScopeInformationAtPosition(document, position);
getScopeInformationAtPosition
getScopeInformationAtPosition(document: LiteTextDocument, position: vscode.Position): Promise<TextmateToken>
Get token scope information at a specific position (caret line and character number).
LiteTextDocument
).vscode.Position
).{Promise<TextmateToken>}
).getScopeRangeAtPosition
getScopeRangeAtPosition(document: LiteTextDocument, position: vscode.Position): vscode.Range;
Get matching scope range of the Textmate token intersecting a caret position.
LiteTextDocument
).vscode.Position
).Promise<vscode.Range>
).getTokenInformationAtPosition
getTokenInformationAtPosition(document: LiteTextDocument, position: vscode.Position): Promise<vscode.TokenInformation>;
VS Code compatible performant API for token information at a caret position.
LiteTextDocument
).vscode.Position
).Promise<vscode.TokenInformation>
).getLanguageConfiguration
getLanguageConfiguration(languageId: string): LanguageDefinition;
Get the active language point configuration of a language mode identifier.
string
).LanguageDefinition
).getGrammarConfiguration
getGrammarConfiguration(languageId: string): GrammarLanguageDefinition;
Get the active language point configuration of a language mode identifier.
string
).GrammarLanguageDefinition
).getContributorExtension
getContributorExtension(languageId: string): vscode.Extension<unknown> | void;
Get the VS Code Extension API entry of the extension that contributed a language mode identifier.
string
).vscode.Extension
).This is the vscode-oniguruma
build of Oniguruma written in C, compiled to WASM format with memory hooks to V8.
This is not streaming 🙁 but vscode
libs must bundle WebAssembly deps so as to support web ecosystem.
import TextmateLanguageService from 'vscode-textmate-languageservice';
const onigurumaPromise = TextmateLanguageService.utils.getOniguruma();