Use an alternative JSON deserialization library

[DanielRosenwasser]

I've heard through the grapevine that one of our current constraints has been JSON size due to the limits of parsing speed. A fairly recent blog post I read actually points out two incredibly capable competitors:

• simplejson
• UltraJSON

See: JSON vs simpleJSON vs ultraJSON

Offhand notes:

• Both libraries seem to be consistently faster for loads.
  • This is significant beacuse we only do loads.
• UltraJSON generally seems to be faster than simplejson.
• UltraJSON has a C code dependency.
• simplejson is written in Python, but has an optional C backend for speed.

I think it would be worth experimenting with the two to see what kinds of gains we can get.

---

I've plotted the data from the blog post below.

Loads

Dumps

[billti]

I took a look at this when trying to improve the perf of the signature help (which was a very large payload). Getting rid of the Python class model (as Zhengbo did for the other payloads above) was the biggest perf win, and made it quite fast.

I did look at some of the other JSON libraries, but the complexity of depending on binary modules (like ujson) seemed excessive - especially with the perf problem not an issue once the above was done.

I'm going to close this for now as resolved/not-an-issue with the class hierarchy gone. We can investigate further if it does still turn out to be a bottleneck on further analysis.

[zhengbli]

Reopen this. As recent logging info shows json serialization / deserialization is still the bottle neck of the performance which sometimes results in unwanted queue timeout on large files.