octomix / josson

A query language for JSON and a template engine to generate text output.
Apache License 2.0
31 stars 8 forks source link

Josson performance #12

Closed ako closed 2 weeks ago

ako commented 11 months ago

Not really broken, but noticed with some initial performance tests that josson is significantly slower than both postgresql json and jstl: https://ako.github.io/blog/2023/08/25/json-transformations.html

raymondchoigit commented 11 months ago

Please try the following query and retest. Thanks!

ServiceData.modelObjects.**.map
(   uid,className,type
    ,query_name:props.query_name.dbValues[0]
    ,query_desc:props.query_desc.dbValues[0]
    ,creation_date:props.creation_date.dbValues[0]
    ,object_string:props.object_string.dbValues[0]
).sort(query_name)
ako commented 11 months ago

Thank for the quick response. Your suggestion makes it slightly faster, but still slower than the alternatives:

raymondchoigit commented 11 months ago

From your testing programs, toPrettyString() is included in the timing of Josson, but not jstl. Please revise and retest. I also want to know their performance. Thanks!

ako commented 11 months ago

PrettyString removed: 793,1020 849,1053 781,1074

PrettyString included: 777,1109 763,1318 765,1362

Thanks!

raymondchoigit commented 11 months ago

I've tried some tests. I found that the first call to function fromJsonString() included the creation of Jackson's ObjectMapper, which is not included in the timing of your testing program for Jslt (new ObjectMapper()). I added the following statement at the beginning of the testing program to pretend that Josson was already used and the required variable instances have been initialized before the performance testing of JSON parsing and transformation. Together with my suggested transformation query statement, that improved the testing result a lot!

Josson.fromJsonString("{}").getNode("0");
ako commented 11 months ago

I am running the tests on windows in parallels. Switching the JDK to microsoft JDK 11 for arm results in the following numbers:

271,300 219,188 212,189

Will also rerun with your suggestions.

ako commented 11 months ago

With your suggestion to preload josson i get the following results: 21,144 16,156 16,166

Indeed much improved. Thanks for the help!

raymondchoigit commented 11 months ago

Thank you!!!

octomix commented 2 weeks ago

Multi-threading capability for array elements manipulation has been added since version 1.4.6 which increase large array processing performance a lot.