Open mw66 opened 2 years ago
OK, I found #118
And the workaround there:
Define white space as _, and
"Hang it off every leaf node of your grammar" ??? !!!
Am I the only one who think this idiom is tedious and ugly?
Is there a parser flag somewhere to let the parser consume / skip white spaces by default?
No.
in most programming languages, using white spaces as separator between different syntactic elements does not need special attention, so do I have to write rule like this:
It actually does need special attention when parsing those programming languages. There's usually a separate "lexing" phase that turns the program into a series of tokens. That will usually abstract away whitespace, though I think it's done more for efficiency than convenience. An LL(1) parser is much more powerful and faster if it can look one token ahead instead of one character ahead.
Parsimonious combines lexing and parsing-of-tokens into a single operation (parsing), which makes the parse trees a bit uglier.
Am I the only one who think this idiom is tedious and ugly?
No, though it's also not that big of a deal in practice. Usually you don't operate on parse trees directly: you can write a few methods in your NodeVisitor
subclass to not care about nodes you don't care about. See e.g. https://github.com/erikrose/parsimonious/issues/114 and some linked issues for some more discussion.
If you'd prefer a more traditional lexing approach, Lark has somewhat similar APIs to Parsimonious, at least for grammar definition, but directly supports lexing and has a syntax for ignoring nodes in the parse tree. Lark also has a bunch of extremely cool features (like integration with hypothesis), so it's worth taking a look at.
The disadvantages of Lark are also fairly clear. CFGs can sometimes be more annoying than PEGs due to ambiguity. And unless your grammar works with an LALR(1) parser, it may be less efficient than parsimonious.
""" Am I the only one who think this idiom is tedious and ugly?
No, though it's also not that big of a deal in practice. Usually you don't operate on parse trees directly:... """
You misunderstood me: I know it won't affect the visitor code, but it makes the grammar definition file ugly: so many useless_
scattering around.
Also, this makes the grammar file locked to parsimonious, when user want to use another PEG parser, the grammar files need to be changed, that is another issue I raised in #191
FYI:
https://github.com/PhilippeSigaud/Pegged/wiki
By default, the grammars do not silently consume spaces, as this is the standard behaviour for PEGs. There is an opt-out though, with the simple < arrow instead of <- (you can see it in the previous example).
I think this <
extension is very convenient.
Hi, I just tried this modified example from the readme:
I'm expecting "bold_stuff" be parsed as text; but I got this error:
in most programming languages, using white spaces as separator between different syntactic elements does not need special attention, so do I have to write rule like this:
Then this is too tedious for any serious programming language grammar definition, and nobody care about the
white_spaces
even if it's parsed.Is there a parser flag somewhere to let the parser consume / skip white spaces by default?
Am I missing something?