Map keys are inconsistently highlighted

[alexjurkiewicz]

Versions

This bug is reproducible in:

• [x] the latest version of the extension (below)
• [x] the latest version of the language server (below)

Extension

v2.22.0

Language Server

0.27.0

VS Code

1.67.1

Operating System

macOS 12.3.1

Terraform Version

Terraform v1.1.9
on darwin_amd64

Steps To Reproduce

locals {
  mymap = {
    key_one           = "foo"
    "key_two"         = "bar"
    (local.key_three) = "baz"
  }
}

Expected Behavior

I would expect key_one and key_two to use the same colour, since they are both strings.

Actual Behavior

Additional context

[radeksimko]

Hi @alexjurkiewicz Thanks for the report.

Given how complex highlighting is I feel it's appropriate to share some context here.

I know you filed this under the vscode-terraform repository, so you probably already know this, but I just want to highlight that this is a Terraform-specific problem. Unquoted map keys are what HCL calls "traversals" (basically) and aside from the obvious edge cases with constants like true or false, these can have different interpretations depending on the language (e.g. Terraform). See https://go.dev/play/p/l4kYwVbHfZg

There are theoretically two ways we could address this.

1. We could update the handling of map keys in the Terraform TM grammar (probably around here), but there's some costs associated with maintaining that level of complexity in a TextMate grammar full of regexes.
2. We could update semantic highlighting rules in the Terraform language server, which in turn uses an HCL generic library hcl-lang - this means we'd also need to give the library some ability to treat map keys differently based on schema/product and then make use of it somewhere here.

Before jumping to solutions though:

• We'll need to discuss this with the Terraform Core team - in part to better understand how/where Terraform itself handles this conversion and also whether there are any best practices or conventions in this area we should be reflecting here.
• We need to understand how other languages outside of Terraform deal with this problem. I know for example that Nomad only allows unquoted map keys - at least in some contexts.
• We need to consider the UX impact - even in Terraform, where the two keys are equivalent, I'm not 100% sure it's more valuable to have them highlighted the same. This again comes back to the question of formatting conventions. e.g. If we prefer quoted keys over unquoted, then I'd argue that users may benefit from those being highlighted differently, such that the preferred formatting stands out next to the alternative one, should users choose to align it all to the same formatting style. Personally I don't have a strong preference here but I do believe that consistency is what we should be striving for (i.e. either quoted or unquoted, but not mix of both).

[alexjurkiewicz]

Yes, that all makes sense. I hope you can make some progress 🙏