Open stephenberry opened 1 month ago
Addresses: #917
Note that partial reading, which is supported for JSON, handles this problem in another manner.
JSON pointer syntax reading also supports this in part. The main difference is that we want a core way to maintain iterators at the parsed location, for the best possible speed. I think this will also take a good step in the direction of handling streams of data.
The new read_allocated
option in #1027 also provides a solution for partial reading.
read_allocated
has been renamed to partial_read
in #1028
The new API design proposal looks nice.
Glaze doesn't easily support incremental reading and writing. There are times when we want to just parse the first element of an array, perform logic in our program, and then decide how what item to use to parse the rest of the array. This is common for formats that need header information.
Another use case is appending data to a buffer incrementally.
This issue proposes the addition of two stateful classes:
gz::reader
andglz::writer
, which will enable incremental reading/writing.Initially this will focus on array support, but object support will be considered in the design.
Example syntax: