denoland / std

The Deno Standard Library
https://jsr.io/@std
MIT License
3.03k stars 606 forks source link

Add support streaming json parsing from array of objects #5675

Open polRk opened 1 month ago

polRk commented 1 month ago

Is your feature request related to a problem? Please describe.

The problem: there is a massive array of objects, and it is very, very large, I want to be able to parse this array in parts, object-wise.

new Response(`[{}, {}, {},\n{},\t{}...]`)
// transform
transformed = ...

for await (const line of transformed) {
   const obj = JSON.parse(line)
}

Describe the solution you'd like

Add transformer that supports top level array or array by path (Eg: {data: [{}, {}, {}]}

Good: https://pkg.go.dev/encoding/json#Decoder, https://pkg.go.dev/encoding/json#example-Decoder.Decode-Stream

Describe alternatives you've considered

The npm ecosystem is full of solutions that are very poorly made or just very slow and resource intensive

BlackAsLight commented 3 days ago

I could have a go at making something like this, but how would it pair with the existing JsonParseStream?