[Feature Request] Semantic highlighting for the various uses of square brackets

nmsmith commented 1 year ago

Request

Square brackets have three different uses in Mojo:

Array literals
Indexing (which can be made available on user-defined data types via __getitem__)
Parameters

I request that Mojo's language server, and its VSCode extension, use semantic highlighting to clearly distinguish each of these meanings.

Motivation

When skimming a Mojo codebase, one is going to see a lot of [..]. Each occurrence will have one of three possible meanings. This could end up being particularly confusing in a region of code that invokes several parametric functions and also does some work with arrays or dictionaries.

Also, the different uses of [..] are likely to cause a lot of confusion for people who are attempting to learn Mojo as a first language. Colorization would help learners realize that they are looking at something different. For example, if a learner has just finished learning about arrays, and array literals were displayed using yellow brackets, they will come to associate arrays with the color yellow. Then, if they later see a type such as Tree[Int], but the brackets are blue, they will hopefully avoid making an incorrect assumption about what those brackets mean. (One problem with this idea: we have no way to guarantee that that learning materials are using semantic highlighting. We can only control the VSCode editor.)

Description and Requirements

Semantic highlighting is a feature defined by the Language Server Protocol, thus semantic tokens can be made available to all IDEs.

The token types that are used to label Mojo's square-bracket tokens might not be understood by the popular color themes. Thankfully, for VSCode in particular, it is possible for an extension to manually specify the default color that should be used for each semantic token type, either for specific themes, or irrespective of the theme. The color specification is done via the extension's package.json:

"contributes": {
    "configurationDefaults": {
        "editor.semanticTokenColorCustomizations": {
            "[<color theme>]": {
                 ...
            }
        }
    }
}

Whoever is working on the VSCode extension will understand what this means.

Addendum

Additionally, it would be good to have semantic tokens (and colorization) for the three uses of parentheses:

Tuple literals (although Python tuples technically don't need to be parenthesised...)
Grouping
Arguments (i.e. ordinary function definitions and calls)

The colorization of the following pairs should probably be the same:

array literals & tuple literals
parameters & arguments

DayDun commented 1 year ago

Great suggestion. Though it does unfortunately clash with how VSCode by default colors matching braces:

Should maybe be toggleable in the extension so you can choose to either get colors based on semantics or based on matching braces.

Or maybe there's a clever compromise where you get the best of both worlds 🤷‍♂️

DayDun commented 1 year ago

Just realized the screenshotted code won't compile 🤦‍♂️

nmsmith commented 1 year ago

Though it does unfortunately clash

That's true: the specific means of introducing the colorization in VSCode would require careful consideration. Though, not everyone uses the (non-semantic) bracket colorization feature seen in your screenshot. (Python programs typically don't involve many nested brackets.) Regardless, as you say, the semantic colorization only needs to be a configurable option. It doesn't need to be imposed upon anyone.

I should emphasise I am proposing two separate things here:

For Mojo's language server to emit semantic tokens for each use of [] and ().
For Mojo's VSCode extension to provide bracket colorization using these semantic tokens.

The first part can be implemented independently of concerns about colorization.

modularml / mojo