kschiess / parslet

A small PEG based parser library. See the Hacking page in the Wiki as well.
kschiess.github.com/parslet
MIT License
805 stars 95 forks source link

Fixes needed for scanning of large files #216

Open tomlea opened 2 years ago

tomlea commented 2 years ago

Fixes to allow my use case of parsing 500mb+ file, with huge blocks of content that do not need parsing, but was taking forever to do without.

kschiess commented 1 year ago

Sounds like a bit of a new use case for parslet; new in the sense that it was never going to work on huge inputs and wasn't made for it - parsing is just too slow. In your case: Would it be impossible to extract parseable bits by other means and then parse the remaining important parts with parslet?

If I merge and then announce parslet works with 500's of MBs, I am putting us in a bad spot...