[ES|QL] revisit field and variable representations

[drewdaemon]

Describe the feature: Column names can be escaped and written in a variety of ways. For example, the following are all valid for event.dataset.

`event.dataset`
"event.dataset"
"""event.dataset"""
`event`.`dataset`

// low priority
`event` . `dataset`
event . dataset
event . /* comment */ dataset

Our support for these in validation and autocomplete is spotty. There are several known bugs that are likely related to this.

• https://github.com/elastic/kibana/issues/191105
• https://github.com/elastic/kibana/issues/191100
• We don't correctly validate dotted field names with separate parts escaped (tracked in https://github.com/elastic/kibana/issues/177699):

  FROM kibana_sample_data_logs | KEEP `geo`.`dest`

There are also things we just can't support today

• I wasn't able to support dotted field names with whitespace in https://github.com/elastic/kibana/pull/190465
• I wasn't able to support escaped columns in https://github.com/elastic/kibana/pull/190466

The ask

Take a comprehensive look at the situation. We have options

• Is the looseness here necessary? Can we tighten the language grammar?
• Would using the AST more in autocomplete help here?
• How are we handling escaping in our cached field and variable lists? It seems like some of the bugs are due to cache misses (e.g. looking for the escaped version of a field when the cache contains the unescaped field name instead).

Some implementations

• https://github.com/drewdaemon/kibana/blob/29cf7576799cc597b9ea5d6ab9778322f49fcfb6/packages/kbn-esql-validation-autocomplete/src/code_actions/actions.ts#L129-L147

[elasticmachine]

Pinging @elastic/kibana-esql (Team:ESQL)

[drewdaemon]

Relevant discussion: https://github.com/elastic/kibana/discussions/182393#discussioncomment-10453762