Implement a streaming GeoJSON reader

[pka]

The current GeoJSON reader is based on https://github.com/georust/geojson which reads the whole file into memory.

A few unsuccessful attempts are in https://github.com/pka/geozero/tree/parse-geojson-with-serde:

• Serde Deserialize with custom deserialize methods for geometries and properties: I was not able to pass the parsed data out of the deserializon methods to the caller
• Serde Transcoder (https://serde.rs/transcode.html): too much semantic information is lost in the serialize methods. Enough for pretty printing, but not well suited for reading complex data structures
• Reading a GeoJSON stream and deserialize the current feature only: I didn't find a reliable way to detect the begin and end of a feature

I also had a deeper look at https://github.com/Geal/nom, which has a streaming interface. But I didn't find a good way to continue after a "Buffer empty" error.

[frewsxcv]

a couple relevant issues in the popular json libraries:

• serde-rs/json#526
• maciejhirsz/json-rust#143

[pka]

Apache Arrow has a batch reader: https://github.com/apache/arrow/blob/master/rust/arrow/src/json/reader.rs

[pka]

Rust library for encoding and decoding JSON streams.: https://docs.rs/destream_json/

[pka]

https://github.com/jeremiah-shaulov/nop-json: Deserialization of JSON io::Read stream into primitive types, Strings and any other types.

EDIT: Currently nighly only.

[pka]

https://github.com/serde-rs/json-benchmark/blob/master/src/canada.rs contains a GeoJSON benchmark.

[bjornharrtell]

Arrow implementation is now at https://github.com/apache/arrow-rs/blob/master/arrow/src/json/reader.rs.

[kylebarron]

Arrow implementation is now at https://github.com/apache/arrow-rs/blob/master/arrow-json/src/reader/mod.rs 🙃

[kylebarron]

Was this issue intended to be closed by #16?

[michaelkirk]

I think so!