goofballLogic / ld-query

Querying JSON-LD
MIT License
20 stars 6 forks source link

Reimplementation of graph walk to optimize performance #24

Closed dthume closed 6 years ago

dthume commented 7 years ago

PnS testing of one of our apps which makes extensive use of ld-query indicated some room for improvement to substantially improve performance and reduce CPU usage on our node servers; this version caches both parsed steps and paths per document, as well as making better use of arrays in the inner loops (push vs unshift / [].concat(old)).

We've tried running with this patch on our in house environment, and are seeing roughly a 50% reduction in CPU consumption overall.

You can test the results locally by checking out this branch and running:

npm run benchmark

which will compare this branch to master

During the implementation of this, I noticed that some of the handling around array valued nodes seemed incorrect; the current master branch will return an array as the result of query if the path traversal ends at a property whose value is an array, but this seems inconsistent with the expected handling of arrays; that each item in the array is one particular solution to the query. I would expect query to always return the first item of such an array (with the exception of @type, which is always known to be an array of strings, never an array of entities).

goofballLogic commented 6 years ago

@dthume, I have invited you (and @van-appears ) as collaborators so you can do merges in future. Do you have an npm username that I can use to make you an admin there?