Add punctuation scopes to parens and brackets

[jwortmann]

I think the contexts not included in the new "expressions" context which is introduced here are safe to never occur within parens or brackets. To allow "preprocessing" being included in "expressions", it was slightly adjusted to not scope a single # on it's own and not scope preceding whitespace (matches the behavior of the C/C++ syntax now).

[eirik-kjonstad]

This looks good! I will try to contribute to adding more meta-scopes soon, but I'll first need to learn more about how to use them properly, as well as how they are commonly used in other syntax definitions.

Should we use meta.parens for all code blocks that fit logically within a parenthesis? I see there are a few places where we match parentheses and enclosed content without adding meta.parens to (...), e.g.

type is (cat)

[jwortmann]

The meta-scopes usually span multiple tokens or are used for bigger sections of the code to provide some context information, for example like we did only for the first line of a function declaration to be able to distinguish the function name used there from the same name after an end function statement, even though both use the same entity.name.function scope. More complex syntaxes for example apply the meta.function scope to the whole function declaration block, or use something like meta.if and/or meta.block for a whole if-block. Sometimes this is a nice-to-have feature, e.g. to ensure correct balancing of { and } for languages which use them to denote blocks. But it requires to always push another context onto the stack at the start of such block statements, which means to carefully include most of the other contexts and rules therein and therefore might be prone to mistakes in case a necessary context has accedentally been missed. On the other hand, some contexts can be deliberately excluded, for example a class definition might not make sense/isn't allowed within an if-block.

Regarding the syntax highlighting, the meta-scopes sometimes allow color schemes to better target specific tokens, but most color schemes don't make much use of it, so usually the meta-scopes don't make any difference. But at least the punctuation scopes for parentheses can be useful, because a color scheme could use a special color for all parens/brackets (e.g. a slightly dimmed foreground color).

Which exact meta-scope to use for (...) could also depend on the context, for example some syntaxes use only meta.function-call as a meta-scope for a function call expression (identifier, parentheses and parameters). But since this syntax currently only matches the identifier of a function call, I think the meta.parens scope for all (...) seems to be sufficient for now (and is a suggested scope for it in the scope naming guidelines). If we'd like to change it to meta.function-call, we would have to adjust the corresponding context to something like this:

function-call:
  - match: '(\w+)\s*(\()'
    scope: meta.function-call.fortran
    captures:
      1: variable.function.fortran
      2: punctuation.section.parens.begin.fortran
    push:
      - meta_content_scope: meta.function-call.fortran
      - match: \)
        scope: meta.function-call.fortran punctuation.section.parens.end.fortran
        pop: true 
      - include: line-continuation
      - include: numbers
      - include: strings
      - include: match-variable
      - include: .....

But as we already noticed before, this example probably isn't even possible or useful in Fortran, because array access and function calls use the exact same syntax.

[eirik-kjonstad]

Thanks for the detailed explanations. I've seen from the documentation pages for Sublime scopes that they don't recommend color-coding meta scopes. I agree with your comments on the possibility for errors when pushing onto new contexts, but also the pros in terms of excluding invalid syntax within e.g. if-statements (such as class definitions).