search

ianprime0509 / zig-xml

XML parser for Zig
http://ianjohnson.dev/zig-xml/
BSD Zero Clause License
17 stars 4 forks source link

Allow `TokenReader` to allocate and introduce reset function #30

Closed ianprime0509 closed 2 months ago

ianprime0509 commented 11 months ago

Right now, TokenReader will only use its internal buffer to store one full token, and after the next call to next, that token is invalidated. This means that clients (such as Reader) have to copy around a lot of data, potentially unnecessarily.

If TokenReader used a growable buffer, it would be possible for it to offload the responsibility of deciding when to reset to the client (e.g. Reader might reset after handling a single event). However, there must still be a way to ensure that normal XML documents do not use unnecessarily large amounts of memory: specifically, there should be a configurable limit on the size of content tokens returned by the reader, so that the client has a chance to reset if desired in the middle of content.

ianprime0509 commented 11 months ago

One outcome which should naturally follow from this is that today's TokenReader should be easily reproducible from this proposed new TokenReader just by providing a FixedBufferAllocator.

ianprime0509 commented 2 months ago

No longer relevant with #36.