pest-parser / pest

The Elegant Parser
https://pest.rs
Apache License 2.0
4.64k stars 260 forks source link

Unexpected behavior: the Silent (_) modifier does not have a cascading effect #457

Open mayrepo opened 4 years ago

mayrepo commented 4 years ago

I tried to implement scheme "#;" comments (described at https://srfi.schemers.org/srfi-62/srfi-62.html) with the following line in the .pest file: NODE_COMMENT = _{ "#;" ~ Node } Node is not silent itself.

But I found that Nodes in NODE_COMMENTs appear in the AST because the Silent (_) modifier does not have a cascading effect.

The doc states that "Silent rules do not create token pairs during parsing, nor are they error-reported." It seems to me that the wording of the doc does not match the actual implementation.

The behavior described in the doc seems more desirable to me (and it matches my use case) so I suggest making the Silent (_) modifier have a cascading effect rather than modifying the doc.

mdg commented 4 years ago

edit: I'm pretty sleepy and think I was seeing things and the below can be ignored. I see it cascading as expected now, sorry for any confusion.

I've found that it sort of cascades. This isn't really helpful, b/c I haven't been able to decipher the exact behavior yet. But in one case, I removed the silent modifier and a bunch of inner rules appeared. What would be great is a silent operator that does cascade and a quiet or transparent operator that only affects that rule and not inner rules, similar to CompoundAtomic.

CAD97 commented 4 years ago

Yeah, this behavior isn't entirely self-evident yet, due to the multiple jobs each modifier holds.

If you don't need automatic whitespace (or can do it manually for the rule), you can achieve a transitively silent rule by writing rule = _{ atomic_rule } and atomic_rule = @{ actual ~ rule }.

This isn't the best workaround, as it requires making the inner rules not eat implicit trivia, but it works... I think. I didn't try it, I'm just going off of memory.