erezsh
commented
5 years ago Sort of, yes.
If you use the LALR parser, and provide it with a Transformer as an argument, Lark will feed it constructs as it goes.
That will allow you the handle them incrementally.
Doing so will still store a big list in the end (of transformed objects), but you can even avoid that, by raising Discard, which will result in an empty tree, if done in every branch.
Given a large stream, is it possible to parse and yield each bit as a generator? For example, https://github.com/danielyule/naya yields a dict for each item in a large json array.