francoispqt / gojay

high performance JSON encoder/decoder with stream API for Golang
MIT License
2.11k stars 112 forks source link

Trouble Unmarshalling Array of JSON Objects #38

Closed alee792 closed 6 years ago

alee792 commented 6 years ago

Hi there,

I'm trying to unmarshal/decode an array of objects, but I'm afraid it's either unimplemented or I'm doing something incorrectly.

Boilerplate for my struct:

type bucketAgg struct {
    IP         string
    Count      int
    ReportTime gojay.EmbeddedJSON
}

func (b *bucketAgg) UnmarshalJSONObject(dec *gojay.Decoder, key string) error {
    switch key {
    case "key":
        return dec.String(&b.IP)
    case "doc_count":
        return dec.Int(&b.Count)
    case "ReportTime":
        return dec.AddEmbeddedJSON(&b.ReportTime)
    }
    return nil
}

func (b *bucketAgg) NKeys() int {
    return 3
}

type bucketChan chan *bucketAgg

func (bc bucketChan) UnmarshalJSONArray(dec *gojay.Decoder) error {
    b := &bucketAgg{}
    err := dec.Decode(b)
    if err != nil {
        return err
    }
    bc <- b
    return nil
}

Test Code:


func TestUnmarshal(t *testing.T) {
    body := []byte(`{"key":"67.13X.XXX.XXX","doc_count":116,"ReportTime":{"value":1.528330747E12,"value_as_string":"2018-06-07T00:19:07.000Z"}}`)
    ba := &bucketAgg{}
    err := gojay.UnmarshalJSONObject(body, ba)
    if err != nil {
        t.Error(err)
    }
    fmt.Printf("%+v\n", ba)
    fmt.Println(string(ba.ReportTime))
}

func TestUnmarshalJsonArray(t *testing.T) {
    var wg = sync.WaitGroup{}
    body := []byte(
        `
[
  {"key":"67.13X.XXX.XXX","doc_count":116,"ReportTime":
    {"value":1.528330747E12,"value_as_string":"2018-06-07T00:19:07.000Z"}
  },
  {"key":"67.25X.XXX.XXX","doc_count":34,"ReportTime":"
    {"value":1.528335168E12,"value_as_string":"2018-06-07T01:32:48.000Z"}"
  }
]
`,
    )
    bc := make(bucketChan)
    wg.Add(1)
    go func() {
        defer wg.Done()
        for r := range bc {
            fmt.Println("Result: ", r)
        }
    }()
    err := gojay.UnmarshalJSONArray(body, bc)
    close(bc)
    wg.Wait()
    if err != nil {
        t.Error(err)
    }
}

The former passes, the latter fails:

=== RUN   TestUnmarshalJsonArray
Result:  &{67.13X.XXX.XXX 116 [123 34 118 97 108 117 101 34 58 49 46 53 50 56 51 51 48 55 52 55 69 49 50 44 34 118 97 108 117 101 95 97 115 95 115 116 114 105 110 103 34 58 34 50 48 49 56 45 48 54 45 48 55 84 48 48 58 49 57 58 48 55 46 48 48 48 90 34 125]}
--- FAIL: TestUnmarshalJsonArray (0.00s)
    .../api_test.go:52: Invalid JSON, wrong char '}' found at position 135
FAIL

I deviated from the docs since all of the array decoding was for standard types and used Decode. I also thought I had a possible issue using the undocumented EmbeddedJSON, but ran into the same problems when dropping that from my struct. I'm guessing that when the current chunk is finished decoding, it expects a ',', but gets the close bracket of the object instead.

Please let me know if I can provide any additional information or if I've made a glaring mistake.

francoispqt commented 6 years ago

Hi, Your JSON is invalid in the test, but still it won't work with valid JSON. When unmarshaling your array you are calling dec.Decode(b) in UnmarshalJSONArray method just change it to dec.Object(b) and it works.

The Decode method should not be used within a unmarshaling method.

Let me know if you have any issue.