search

yhirose / cpp-peglib

A single file C++ header-only PEG (Parsing Expression Grammars) library
MIT License
884 stars 112 forks source link

token() in AST different from SemanticValues #127

Closed sasq64 closed 3 years ago

sasq64 commented 3 years ago

It seems token() will returned the matched value from SemanticValues at any part in the tree, but in an AST it will only return something for the leaf nodes. I don't know if this is intentional but it would be very practical to be able to access the matched text in non leaf nodes.

yhirose commented 3 years ago

@sasq64, thanks for the feedback. this is on purpose to reduce memory usage in an AST tree. In my usages so far, there is no need to access token text on non leaf nodes at all. In the next cpp17 version, we could extend it to non leaf nodes since the token member is now std::string_view that doesn't copy any source text.

sasq64 commented 3 years ago

Incidentally I'm using the cpp17 branch so I didn't consider the fact that it was a space issue... I'm recreating the contents by mean of position/size and the original source now instead.