search

pointlander / peg

Peg, Parsing Expression Grammar, is an implementation of a Packrat parser generator.
BSD 3-Clause "New" or "Revised" License
1.02k stars 120 forks source link

using peg with io.RuneReader #125

Closed amery closed 3 years ago

amery commented 3 years ago

hi, could you explain how to use peg generated code against text coming from a stream? if not supported, could you outline how you would want this implemented?

pointlander commented 3 years ago

I would recommend reading and saving symbols from the stream until you find the END token for whatever you are processing and then feeding the accumulated text into peg. Because peg is a backtracking parser, parsing a stream directly doesn't make much sense because the text from a stream must be buffered anyways.

amery commented 3 years ago

as the stream doesn't necessarily have an END I need to be able to identify and process complete messages and postpone the incomplete one until more data is collected... without having to wait until the end of times to start the work, and without having to write a separated parser manually to pre-split the stream...

pointlander commented 3 years ago

This is the downside of having a parser that can backtrack. It can backtrack to the beginning of time, so finding the chunks of input is left up to the user of the parser.

amery commented 3 years ago

thanks, I understand now