Upcoming when clause context parser

ulugbekna commented 1 year ago

What's a when clause?

When clauses allow to selectively enable and disable various extension contributions such as commands and UI elements (e.g., menus or views). When clauses are simple expressions that are put in an extension’s manifest (i.e., package.json).

New parser

The next VS Code release is planned to include a new parser of when clauses. The new parser will come with new features, more precise but stricter parsing rules, and a linter to see incorrect when clauses right away in package.json files.

Below we outline new features, breakages, and a way to make sure your extension works as intended. Please, subscribe to this issue to stay up-to-date with other changes that may happen before the next VS Code release 1.77.

New features

Support for parentheses

(requested in https://github.com/microsoft/vscode/issues/91473)

Note: Negation of expressions such as !(editorTextFocus && debugState == 'running') is still work-in-progress. A comment will be posted to this issue, when it’s ready. Currently, one can only negate a context key, e.g., !editorTextFocus.

Negation of expressions (e.g., !(editorTextFocus && debugState == 'running')) is also now supported.
Support for more regular expression flags

Key-value when clause operator =~ currently supports the i flag for case-insensitive regular expression search, but the new parser will support the following flags are supported: i, s, m, u. VS Code will not throw an error if g and y flags are put in the regular expression but will not use them.
Support operators === and !==

The new parser allows === and !== operators, but they behave the same as == and !=. We noticed that there are already extensions that use these operators, assuming that they are supported, so we designed the new parser to be more flexible and support such operators.

Breakages

We tried our best to keep old parser’s behavior on most when clauses so that existing when clauses work as they did before. There are three cases to keep in mind, however, for a when clause to work as intended:

When used with key-value when clause operator, a regular expression needs to be put between slash characters / just as in JavaScript, e.g., view =~ /[a-z]+/.

Also important to note that a slash character within a regular expression needs to be escaped with two backslash characters, e.g., "when": "workspacePath =~ /my\\/path\//" . The first backslash is to escape the second backslash in the JSON string within package.json and the second backslash is to escape the slash character.
A string value containing whitespace must be quoted, e.g.,

"when": "myContextKey == 'two words'"

Note: only single quotes are currently supported.
Comparison operators need to be separated by whitespace from its operands, i.e.,

correct: count < 2

incorrect: count<2

Migration

VS Code Insiders comes with a linter for when clauses when an extension manifest file package.json is open. See no errors are reported for your extension’s package.json.

VS Code Insiders will also run the new parser by default to see how extensions work with the new parser -- a comment will be posted when this happens as a notification to check out how your extension works with it.

Latest parsing grammar

In EBNF.

expression ::= or

or ::= and { '||' and }*

and ::= term { '&&' term }*

term ::=
    | '!' (KEY | true | false | parenthesized)
    | primary

primary ::=
    | 'true'
    | 'false'
    | parenthesized
    | KEY '=~' REGEX
    | KEY [ ('==' | '!=' | '<' | '<=' | '>' | '>=' | 'not' 'in' | 'in') value ]

parenthesized ::=
    | '(' expression ')'

value ::=
    | 'true'
    | 'false'
    | 'in'          
    | VALUE 
    | SINGLE_QUOTED_STR
    | EMPTY_STR

Please, let me know if you have any feedback! :-)

bwateratmsft commented 1 year ago

Will "when": "never" still continue to work equivalently to "when": "false"?

Eskibear commented 1 year ago

AFAIU, "never" is not a keyword or a pre-defined constant. Currently it's equivalent because context key "never" is undefined. Technically I can run vscode.commands.executeCommand("setContext", "never", true) in any extension to break the assumption.

ulugbekna commented 1 year ago

Hi @bwateratmsft,

@Eskibear is right, never is interpreted just as a context key. So unless anyone sets a value to it, it evaluates to false :-)

ulugbekna commented 1 year ago

Hello everyone,

[x] The latest VS Code Insiders runs with the new parser as the default one to parse context key expressions, so you should be able to test your extensions.
[x] The latest Insiders also includes negation of arbitrary expressions, which means that you can now negate parenthesized expressions, e.g., "when": "foo && !(bar || baz)".

I also updated the EBNF grammar in this issue's description above.

Please, let me know if you have any feedback! :-)

starball5 commented 1 year ago

In preparation for any future breakages related to extensions or user-local keybindings, I have created a Q&A on Stack Overflow, which you can (where appropriate) use as a duplicate-target. A fair number of people often go to Stack Overflow with problems like this. Feel free also to suggest edits to the question and answer posts to improve them.

Here it is: Why are some of my extensions or keybindings' when clauses broken in VS Code 1.77's official and insiders releases?

ulugbekna commented 1 year ago

The new parser now runs by default in VS Code 1.77 (stable).

Thanks everyone!

I will keep this issue around in case there are any future changes to the parser.

ian-h-chamberlain commented 1 year ago

VS Code Insiders comes with a linter for when clauses when an extension manifest file package.json is open. See no errors are reported for your extension’s package.json.

Is there any plan to port this to the user settings keybindings.json? When setting up some new keybindings I accidentally left a when clause with a trailing && in it, and it seems to just evaluate to true, without any indication of an error that I could see. I'm running stable VSCode 1.80.1 so I assume it should be using this new parser, correct?

Should I file a separate issue for that, or is this issue the right place to report that?

ulugbekna commented 1 year ago

@ian-h-chamberlain made a request https://github.com/microsoft/vscode/issues/188427. Thanks for the heads up 😊

jimasp commented 1 year ago

One thing I really feel is missing is the ability to inject a regex context string, e.g.

extension.js

const myRegexWildcardPaths = myFolderPrefixList.map(f => `${f}.*`).join("|");
vscode.commands.executeCommand('setContext', 'extensionContext.myRegexWildcardPaths', myRegexWildcardPaths);

package.json:

"when": "resourcePath =~ /extensionContext.myRegexWildcardPaths/ && !isInEmbeddedEditor && editorTextFocus"

ian-h-chamberlain commented 3 days ago

I will keep this issue around in case there are any future changes to the parser.

Has something changed in a more recent version w.r.t. to regex parsing? In particular as it relates to this, my regex matching seems to require quotes now instead of slashes:

When used with key-value when clause operator, a regular expression needs to be put between slash characters / just as in JavaScript, e.g., view =~ /[a-z]+/.

Keybindings that used to work for me e.g. neovim.fullMode =~ /^(i|n[or])/ now appear only to work when I use quotes like neovim.fullMode =~ '^(i|n[or])' instead, which is fine (since there is a way to fix it) but surprising given the requirement written above. Should I file a new bug for this?

Edit: system details

Version: 1.95.3 (user setup)
Commit: f1a4fb101478ce6ec82fe9627c43efbf9e98c813
Date: 2024-11-13T14:50:04.152Z
Electron: 32.2.1
ElectronBuildId: 10427718
Chromium: 128.0.6613.186
Node.js: 20.18.0
V8: 12.8.374.38-electron.0
OS: Windows_NT x64 10.0.19045

microsoft / vscode