search

amazon-ion / ion-docs

Source for the GitHub Pages for Ion.
https://amazon-ion.github.io/ion-docs/
Apache License 2.0
22 stars 22 forks source link

Comments as Whitespace vs Stop Characters #314

Open rmarrowstone opened 2 months ago

rmarrowstone commented 2 months ago

There is some ambiguity between the spoken-language spec and the Text Grammar about whether comment starts delimit the end of "numeric_entities" and "sexp_keyword_entities."

Consider the below:

2004T/* comment after Timestamp with no whitespace*/

Per the grammar, that should be legal as comments are included in ws.

top_level_value
    : annotation+ top_level_value
...
    | numeric_entity ws // timestamp is a "numeric_entity"
... 

ws
    : WHITESPACE
    | INLINE_COMMENT
    | BLOCK_COMMENT
    ;

But per the spoken-language spec, '/' is not a "stop-character."

In the text notation, timestamp values must be followed by one of the fifteen numeric stop-characters: {}[](),\"\'\ \t\n\r\v\f

This same ambiguity exists for integers, real numbers and keywords.

The ion CLI considers following any of the above a parse error. I have not canvassed other implementations.