denoland / std

The Deno Standard Library
https://jsr.io/@std
MIT License
3.22k stars 621 forks source link

Add support streaming json parsing from array of objects #5675

Open polRk opened 3 months ago

polRk commented 3 months ago

Is your feature request related to a problem? Please describe.

The problem: there is a massive array of objects, and it is very, very large, I want to be able to parse this array in parts, object-wise.

new Response(`[{}, {}, {},\n{},\t{}...]`)
// transform
transformed = ...

for await (const line of transformed) {
   const obj = JSON.parse(line)
}

Describe the solution you'd like

Add transformer that supports top level array or array by path (Eg: {data: [{}, {}, {}]}

Good: https://pkg.go.dev/encoding/json#Decoder, https://pkg.go.dev/encoding/json#example-Decoder.Decode-Stream

Describe alternatives you've considered

The npm ecosystem is full of solutions that are very poorly made or just very slow and resource intensive

BlackAsLight commented 2 months ago

I could have a go at making something like this, but how would it pair with the existing JsonParseStream?