thedevsaddam / gojsonq

A simple Go package to Query over JSON/YAML/XML/CSV Data
https://github.com/thedevsaddam/gojsonq/wiki
MIT License
2.17k stars 140 forks source link

Cannot read json stream using Reader() #105

Open EwanMe opened 1 year ago

EwanMe commented 1 year ago

I'm trying to read JSON from a stream using the New().Reader() method (I'm using the docker api). I want to read the same JSON value which will be continuously updated in a for loop. Using the following code my output is empty and my program will just hang in the loop forever.

resp, err := cli.ContainerStats(ctx, cont_id.String(), true)
if err != nil {
    panic(err)
}

jq := gojsonq.New().Reader(resp.Body)

if jq.Error() != nil {
    panic(jq.Errors())
}

for {
    value := jq.Find("myKey")
    fmt.Println(value)

}

For the sake of the example my JSON is just:

{
  "myKey": "myVal"
}
thedevsaddam commented 1 year ago

Please try Reset() method, value := jq.Find("myKey").Reset()

EwanMe commented 1 year ago

Maybe I'm misunderstanding, but the return value of Find("myKey") is just an interface no? So it has no method Reset(). However, I tried calling jq.Reset() inside the loop, but with no effect. I believe the problem is that the JSONQ.Reader() calls ReadFrom() on resp.Body (in the case of my example). ReadFrom() seems to read until EOF which the stream I'm giving as input will not do until it is manually stopped or crashes.

I think the implementation of Decoder.Decode() from encoding/json uses an approach where the buffer is parsed one element at a time, and not until EOF. I managed to get my desired result with the following snippet. Could a similar feature be considered added to gojsonq?

dec := json.NewDecoder(containerStats.Body)

var stats customStructForDockerStats
for dec.More() {
    err := dec.Decode(&stats)

    if err != nil {
        panic(err)
    }
    fmt.Println(stats.myVal)
}