tidwall / gjson

Get JSON values quickly - JSON parser for Go
MIT License
14.1k stars 846 forks source link

[Feature request] Is it possible to add y.GetMany() for x.Array() #233

Closed petric3 closed 2 years ago

petric3 commented 2 years ago

I have a problem similar to this:

------ file xy.log with lots of lines-------
[{"str1":"hello","value1":1.2,"value2":3.14,"time":12345678912},{"str1":"world","value1":2.1,"value2":4.43,"time":12345678913}]
[{"str1":"hello","value1":1.2,"value2":3.14,"time":12345678912},{"str1":"world","value1":2.1,"value2":4.43,"time":12345678913},{"str1":"dinner","value1":12.1,"value2":24.43,"time":12345678914}]
[{"str1":"hello","value1":1.2,"value2":3.14,"time":12345678912},{"str1":"world","value1":2.1,"value2":4.43,"time":12345678913}]

So far I've used it like this:

for lines.Scan() {  // bufio lib
     g = gjson.Parse(lines.Text())
     for _, r := range g.Array() {
          str := r.Get("str1").String()
          value = r.Get("value1").Float()
          // some  code
     }
}

It would be awesome to have res:=r.GetMany("str1","value1", etc) for g.Array() similar to gjson.GetMany(). I assume, this way it would be also faster than extracting each key via r.Get(), t.i. going via a line only one time than nb of r.Get() times (?)

Thanks a lot for the great library, it's super fast :)

tidwall commented 2 years ago

You can use gjson.GetMany with use r.Raw like:

results := gjson.Get(r.Raw, "str1", "value1")

Another option is to use gjson to iterate over the lines and array values like:

gjson.ForEachLine(json, func(line gjson.Result) bool {
    line.ForEach(func(_, entry gjson.Result) bool {
        res := gjson.GetMany(entry.Raw, "str1", "value1")
        // some code using res[0] and res[1]
        return true
    })
    return true
})
petric3 commented 2 years ago

Thank you for the answer. 

Interestingly, I benchmarked gjson.GetMany() vs gjson.Get() and gjson.Get() is faster for about 15% although intiutively I'd say gjson.GetMany() should be faster as gjson.Get() has more calls than gjson.GetMany():

res := gjson.GetMany(r.Raw, "name1", "name2", "value1", "value2")
name1 = res[0].String()
name2 = res[1].String()
value1 = res[2].Float()
value2 = res[3].Float()

vs

name1 = r.Get("name1").String() 
name2 = r.Get("name2").String() 
value1 = r.Get("value1").Float()    
value2 = r.Get("value2").Float() 

Regarding the gjson.ForEachLine, it's slower for my application than bufio due to string conversion of a loaded file.

Thank you once more  🙏

tidwall commented 2 years ago

You're welcome :)