itchyny / gojq

Pure Go implementation of jq
MIT License
3.3k stars 119 forks source link

Can gojq be used in a streaming way? #232

Closed emaxx-google closed 11 months ago

emaxx-google commented 11 months ago

Can this Go library be used without having the whole JSON read and parsed in advance?

E.g., https://github.com/bcicen/jstream allows parsing JSON in a streaming way. Would it be possible for gojq to do a streaming processing in a similar way?

wader commented 11 months ago

Maybe can copy or be inspired by https://github.com/itchyny/gojq/blob/main/cli/stream.go and look how the cli code uses it with the library? it's the code that implements --stream

$ go run cmd/gojq/main.go -c --stream <<< '[{"id": 0, "desc": "RGB"}, {"id": 1, "desc":"cymk"}]'
[[0,"id"],0]
[[0,"desc"],"RGB"]
[[0,"desc"]]
[[1,"id"],1]
[[1,"desc"],"cymk"]
[[1,"desc"]]
[[1]]
itchyny commented 11 months ago

Yes, Go standard library provides streaming lexer, so pass the token value to gojq.