Open david-wahlstedt opened 3 months ago
Didn't really test this, but you could try:
character_class_atom
: quoting // <-- added
| character_class_range
| posix_character_class
| character
| character_type
// | '\\' . // <-- removed
| ~( '\\' | ']')
;
...
quoting
: '\\' 'Q' .*? '\\' 'E' // <-- swapped with the alt below
| '\\' .
;
Thanks! It works!
I am implementing a PCRE2 parser using BNFC, a Haskell based tool. One provides an LBNF grammar, and gets a data type, a parser, a pretty printer, and a case skeleton for the type. I don't know if BNFC is the best choice, but it's fun to try. It is quite challenging, and I have looked at your grammar to help understand the grammatical structure of PCRE, along wiht the man pages. There is not much formal information out there about the grammar of PCRE: your grammar is the only one I've found. Nice work, thanks! I tried your examples, and I see some of the examples, e.g. in various.txt are not valid according to regex101.com. I can send more feedback on that later!
To provoke an error I tried \Qd\Ec\E
on your patched parser, and it gave
echo -n '\Qd\Ec\E'|antlr4-parse PCRE.g4 pcre -tree
(pcre:1 (alternation:1 (expr:1 (element:1 (atom:19 (quoting:1 \ Q d \ E))) (element:1 (atom:15 (letter:1 c))) (element:1 (atom:19 (quoting:2 \ E))))) <EOF>)
The exit status was 0 and I don't see any error message. The \E
in the end should trigger an error, I think, at least regex101.com does. But pcre2test does not: it seems a bit forgiving. I wonder if there is any reference tool that can really check whether or not a PCRE/PCRE2 expression is valid or not? (sorry for my too long answer)
David
It's been quite a while, but I think I wrote the grammar "forgivingly" as well, making it the responsibility of the semantic phase to report any incorrect tokens (like the dangling \E
). Good to hear the grammar is of help to you. Best of luck!
Yes, it makes sense to leave some parts to the semantic side: especially when so much depends on which options are set, version, etc. Thanks!
I tried the PCRE.g4 grammar on an example where the bounds in a character class are quoted sequences. The parser took
\E
and\Q
as bounds, when it should have been the minimal element in the first quoted sequence (a
) and the maximum in the second (d
). I also tried to have only one of the bounds as a quoted sequence with similar result.This is how it behaves with pcre2test: (I also tried it for PCRE (ver 1) using https://regex101.com/ with correct result)
I understand that the example is weird, but it could perhaps be handled differently, like letting the
\Qba\E
block be a possible value as range endpoint, and then let the semantics take care of what it means.I also have a question I don't know where to put: are you going to support PCRE2 any time soon? This is the only formal grammar I have found out there for PCRE. Great job!!!
Best regards, David