tirumaraiselvan / graphql-engine

Blazing fast, instant realtime GraphQL APIs on Postgres with fine grained access control
https://hasura.io
Apache License 2.0
2 stars 0 forks source link

Use streaming JSON parser for combining JSON results #21

Open chrisdone opened 5 years ago

chrisdone commented 5 years ago

Currently we're using aeson's decode to parse into Value structures, lookup the data field, then union the HashMaps, which is convenient but less efficient than if we parsed the boundaries of the object using the attoparsec combinators, located the data key boundaries, and then just concatenated the bytestrings together at the right point as a Builder (the EncJSON uses this). Here is an example of parsing directly.

jberryman commented 5 years ago

I just feel icky doing performance work when we don't have benchmarks. It's first of all hard to tell if the effort is worth it, and second we don't get to feel good about any big gains we've attained.

@tirumaraiselvan do you have any thoughts on benchmarking as it relates to this feature?

also relevant to https://github.com/tirumaraiselvan/graphql-engine/issues/30

jberryman commented 5 years ago

I can take this one.

tirumaraiselvan commented 5 years ago

Yeah, these are optimizations that we better do more systematically. So let's put this on hold.