search

sublimehq / Packages

Syntax highlighting files shipped with Sublime Text and Sublime Merge
https://sublimetext.com
Other
2.95k stars 587 forks source link

[RFC] Key-Value Separators #3228

Open deathaxe opened 2 years ago

deathaxe commented 2 years ago

Relational

Several syntaxes such as Python, JavaScript or JSON support data objects in the form of arrays, lists or key-value pairs.

Those data objects can be grouped into:

  1. sequences
    • arrays (C, C++, ...)
    • lists (Python, JS, JSON, ...)
    • vectors (C++ std/boost library)
    • ...
  2. mappings (aka. key-value pairs)
    • dictionaries (VBScript)
    • maps (C++ std/boost library)
    • objects (JSON)
    • ...

That's pretty clearly expressed by meta scopes meta.sequence and meta.mapping.

List separators are commonly scoped punctuation.separator.sequence.

["item1", "item2", ...]
//      ^ punctuation.separator.sequence

The situation with mapping punctuations seems a bit more unclear at the moment which caused in several solutions having found their way into the wild.

{"key1": "value1", "key2": "value2", ...}
//     ^ punctuation.separator.???
//               ^ punctuation.separator.???

Motivation

The current situation is reflected by a couple of discussions started in pending PRs, such as https://github.com/sublimehq/Packages/pull/2682#issuecomment-964466320 or https://github.com/sublimehq/Packages/pull/2564#discussion_r788000603 which re-introduces punctuation.separator.mapping to YAML, while it has been removed by https://github.com/sublimehq/Packages/commit/60e7c391deb32d1d39b5add199f827c7d3bb8050.

If this topic has been discussed somewhere the related posts seem lost. Hence it may make sense to come up with a guideline how to commonly scope punctuations in data objects such as mappings or sequences in a (this) RFC issue.

Current Situation

The following table, started by @jwortmann, illustrates currently used scopes for separators of key-value pairs.

Syntax key-value separator, e.g. : or = entry separator, e.g. , or ; uses meta.mapping
BibTeX punctuation.separator.key-value punctuation.separator.mapping.pair :heavy_check_mark:
Cabal (PR) punctuation.separator.mapping.key-value - :heavy_check_mark:
CSS punctuation.separator.key-value punctuation.terminator.rule :x:
D punctuation.separator.mapping.key-value punctuation.separator.sequence :x:
Erlang punctuation.separator.mapping.key-value punctuation.separator.mapping.pair :heavy_check_mark:
JSON punctuation.separator.mapping.key-value punctuation.separator.mapping.pair :heavy_check_mark:
Java Properties punctuation.separator.mapping.key-value - :heavy_check_mark:
Git config keyword.operator.assignment - :heavy_check_mark:
JavaScript (object literal) punctuation.separator.key-value punctuation.separator.comma :heavy_check_mark:
Lua (table)* punctuation.separator.key-value punctuation.separator.field :heavy_check_mark:
Ruby punctuation.separator.key-value punctuation.separator.sequence :x:
Perl punctuation.separator.key-value punctuation.separator.sequence :x:
Python (dict) punctuation.separator.mapping.key-value punctuation.separator.mapping :heavy_check_mark:
YAML punctuation.separator.key-value.mapping - :x:

Notes:

The concluding question is about priority of key-value vs. mapping or about which of both the more general part of a scope name is.

Both scope naming schemes punctuation.separator.key-value and punctuation.separator.mapping are both used with nearly the same amount, while the latter one found its way into syntaxes by being suggested by PackageDev's auto-completions.

Suggestion

We should probably condence those scopes to ...

Reasons

  1. Not all syntaxes know about the concept of dictionaries/objects/mappings but still need a way to separate property keys from values.
  2. As a result of 1. punctuation.separator.key-value seems the more general scope which color schemes can use to target most possible tokens with just one simple selector.
  3. If it seems important to distinguish key-value separators of mappings from others meta.mapping punctuation.separator.key-value or `punctuation.separator.key-value.mapping may be used.
  4. A dictionary/object/mapping can be interpreted as a list of key-value pairs. Hence punctuation.separator.sequence seems suitable to separate key-value pairs.
  5. To further justify punctuation.separator.sequence - both, lists and objects often use the same character (e.g.: comma ,) to separate list items and key-value pairs (see: JSON). A common scope ensures both to be highlighted the same way by all color schemes. Further distinctions can be made by eighter meta.mapping punctuation.separator.sequence - meta.mapping meta.sequence or punctuation.separator.sequence.mapping if really needed.

Alternative

Distinguish between real data lists/objects and other kinds of key-value pairs, which means to keep punctuation.separator.mapping.key-value and punctuation.separator.key-value.

jrappen commented 2 years ago

I find your suggestion of using two scopes:

vs. three scopes:

confusing when we have a situation within meta.mapping where:

which would leave us with:

{ "key1": "value1", "key2": "value2", ... }
//      ^ meta.mapping punctuation.separator.key-value
//                ^ meta.mapping punctuation.separator.sequence

vs.:

{ "key1": "value1", "key2": "value2", ... }
//      ^ meta.mapping punctuation.separator.mapping.key-value
//                ^ meta.mapping punctuation.separator.mapping.pair

Or are you suggesting :question:

Is the main reason for this suggestion to limit scopes to three levels?

Random fact of the day: Emojis originate from Japan where they use a combination of :o: and :x: vs. :heavy_check_mark: and :x:.

jwortmann commented 2 years ago

Is the main reason for this suggestion to limit scopes to three levels?

From my point of view the main reason is to standardize a single scope name which can be used in all syntaxes, to make it easier for color schemes to target the key-value separators. As it was pointed out in the linked commit and previous discussions, "key-value" is more general than "mapping" at the third level of the scope name. The scope punctuation.separator.key-value is already used in a few syntaxes where it is not part of an actual mapping literal, for example in HTML:

<div class="foo">
<!--      ^ punctuation.separator.key-value - meta.mapping -->

For this reason we can't use punctuation.separator.mapping if we want to unify the scope names. And if you want to only target the separator punctuations which are inside of actual mapping literals, then it should be sufficient to use the selector meta.mapping punctuation.separator.key-value in color schemes.

When I looked at the Lua tables1 recently, I also had the idea about using punctuation.separator.sequence for the comma even inside of mappings, but I didn't suggest it because I thought it was too big of a change and might seem a bit confusing on first sight. But now I like it, because a mapping is just a sequence of key-value pairs (the meta.mapping scope is of course still kept to make this distinction possible). And it would make it even easier to target all the commas (or similar punctuation symbols) in both mappings and array structures at the same time, with only a single scope. If you want to target the commas in those two kind of structures differently, then you can use two separate rules and prepend the corresponding meta-scope for each rule.

The only drawback I see, is that the statement from the last sentence doesn't work for certain nested combinations of both mappings and arrays, because of the bug2 described in https://github.com/sublimehq/sublime_text/issues/2152. This is probably only relevant for JSON, and I would think that the occasion that someone wants to target the commas in only one of mapping or array seems extremely unlikely and is negligible. Besides that, this "bug" should be fixed in ST core and shouldn't stop us from introducing consistent scope names.

Therefore I would say that appending a further mapping at the fourth level after key-value/sequence to the punctuation scope would be superfluous; we already have the meta.mapping.

(1) Afaik Lua tables are both mapping and array in one, you can use them like {"foo", "bar", "baz"} with normal array indexing (with consecutive index numbers assigned automatically for the keys), or add the keys yourself like {key1="foo", key2="bar", "baz", "foobar"}, even in mixed form.

(2)

>>> sublime.score_selector("meta.mapping meta.sequence meta.mapping punctuation.separator.sequence", "meta.mapping punctuation.separator.sequence")
12304
>>> sublime.score_selector("meta.mapping meta.sequence meta.mapping punctuation.separator.sequence", "meta.sequence punctuation.separator.sequence")
12416
jrappen commented 2 years ago

Ok, I was confused for a bit because adding meta.mapping wasn't part of the suggestion.

jrappen commented 2 years ago

I've added the requested changes to JSON. Will update #3097 soon accordingly.