pest-parser / pest

The Elegant Parser
https://pest.rs
Apache License 2.0
4.63k stars 259 forks source link

Minor inconsistency between pest meta grammar and some tools #853

Open Tartasprint opened 1 year ago

Tartasprint commented 1 year ago

Describe the bug There is a difference between how pest_meta and the other tools/libraries handle the ' in range expressions. In meta/src/grammar.pest the following is a valid range expression '''..''' as there is no check for it: https://github.com/pest-parser/pest/blob/7f8411a4d5b9c9ac6fcec06e293878982cd320c8/meta/src/grammar.pest#L167-L179 There is no check in the meta's parser either:

https://github.com/pest-parser/pest/blob/7f8411a4d5b9c9ac6fcec06e293878982cd320c8/meta/src/parser.rs#L428-L436

Meanwhile other tools do not accept this syntax. The highlighter for the playground has this line for handling ranges:

{ regex: /'(?:[^'\\]|\\(?:[nrt0'"]|x[\da-fA-F]{2}|u\{[\da-fA-F]{6}\}))'/, token: "string" },

And a similar rule seem to be defined for IDEs syntax highlighting.

Searching for usage the unescaped version ''' in public GitHub repositories gave no results, and some results for the escaped version '\''.

To Reproduce Writing the following pest grammar in a file using a supported IDE or in the playground:

r={'''..'''}

Is deemed valid by Pest, the rule r matches the input ', but breaks the syntax-highlighting. Might break other things, I didn't check.

Expected behavior I expected the behavior of the different tools listed up there to be consistent.

Additional context I don't know if ''' being valid was a design choice that wasn't followed in the playground/ide tools, or if checking for this edge case was simply forgotten in pest meta grammar. I'm aware this "bug" doesn't have a great impact (maybe none?), but still I think it should be addressed. I'd happily submit changes to address it, just not sure if that is wanted.

tomtau commented 1 year ago

I don't know if ''' being valid was a design choice that wasn't followed in the playground/ide tools, or if checking for this edge case was simply forgotten in pest meta grammar.

I'd guess the latter? @dragostis @CAD97 @pest-parser/triage do you know?