Open polRk opened 3 months ago
Is your feature request related to a problem? Please describe.
The problem: there is a massive array of objects, and it is very, very large, I want to be able to parse this array in parts, object-wise.
new Response(`[{}, {}, {},\n{},\t{}...]`) // transform transformed = ... for await (const line of transformed) { const obj = JSON.parse(line) }
Describe the solution you'd like
Add transformer that supports top level array or array by path (Eg: {data: [{}, {}, {}]}
{data: [{}, {}, {}]}
Good: https://pkg.go.dev/encoding/json#Decoder, https://pkg.go.dev/encoding/json#example-Decoder.Decode-Stream
Describe alternatives you've considered
The npm ecosystem is full of solutions that are very poorly made or just very slow and resource intensive
I could have a go at making something like this, but how would it pair with the existing JsonParseStream?
JsonParseStream
Is your feature request related to a problem? Please describe.
The problem: there is a massive array of objects, and it is very, very large, I want to be able to parse this array in parts, object-wise.
Describe the solution you'd like
Add transformer that supports top level array or array by path (Eg:
{data: [{}, {}, {}]}
Good: https://pkg.go.dev/encoding/json#Decoder, https://pkg.go.dev/encoding/json#example-Decoder.Decode-Stream
Describe alternatives you've considered
The npm ecosystem is full of solutions that are very poorly made or just very slow and resource intensive