UpstandingHackers / hammer

Parser combinators for binary formats, in C. Yes, in C. What? Don't look at me like that.
GNU General Public License v2.0
430 stars 40 forks source link

[Wiki/User guide] explain use in stream parsers #187

Closed c-alpha closed 1 year ago

c-alpha commented 6 years ago

The user guide in the wiki is an excellent intro, and it has helped me a lot. I would like to suggest some topics to further elaborate:

  1. Backend selection and impact It seems that the backend is selected automatically for each HParser based on its grammar? At least I couldn't seem to find any hints as to how any backend would be selected manually. For the avoidance of doubt: I do not want to select backends, I think that would be a bad idea. Reason for asking is that I tried to use h_parse_start(), h_parse_chunk(), and h_parse_finish(). When h_parse_start() returned NULL, I checked hammer.h and found a cunningly placed comment saying "This is only supported by some backends." ;-) So it seems some explanatory text would be useful, which explains under which circumstances h_parse_start(), h_parse_chunk(), and h_parse_finish() can be used.

  2. Memory management The "Fun with parse results" section in principle already says it, but I would opine that the memory management topic could benefit from some more prose. I would e.g. propose to explicitly state that an HParseResult contains a complete tree of all parsed data. If you parse some non-trivial amount of data, some quite large memory allocations can result. I e.g. parsed some 19MB with a single h_parse() call, and ended up with close to 7GB of allocated memory. In hindsight, this is of course what was to be expected, but IMO it would be helpful if a few words on this aspect could be shed in the user guide.

Both of the above points could for instance be covered in a new "event-based stream parsing" section of the user guide, which outlines how to use hammer in such a scenario.