bjouhier / i-json

Fast incremental JSON parser
38 stars 5 forks source link

Materialization of individual properties #2

Closed mogill closed 10 years ago

mogill commented 10 years ago

Materialization of the entire JSON object is sometimes unnecessary because only a few fields are accessed. For example, a long list requiring 15ms to parse may be accessed conditionally based on other properties of the object. If the properties can be materialized independently, the expensive parsing operation can be deferred until it is known if the data is actually required.

Do you have a sense of how difficult it might be to modify i-json to materialize only a requested property (perhaps via a V8 accessor or interceptor) instead of returning the entire object?

bjouhier commented 10 years ago

You can do this by testing the path in your callback and always returning undefined from the callback. If your callback always returns undefined, the parser will only allocate empty objects and empty arrays for the intermediate object/arrays but it will not set their properties/elements so it won't materialize the tree. In this configuration the parser behaves very much like a pure evented parser.

Another interesting case is when parsing a very large array of objects (like a JSON log stream). In this case you can setup the callback with a depth of 1 and return undefined from the callback. Your callback will receive the objects (the subtrees) as they get parsed but the objects won't be added to the result array. So you will operate with a constant memory footprint even if you parse a very large file.