search

javaee / json-processing-spec

Legacy JSON Processing spec. Please use the link below to find the current JSON P project
https://github.com/jakartaee/jsonp-api
Other
8 stars 3 forks source link

Add Support for processing big JSON data #70

Closed glassfishrobot closed 7 years ago

glassfishrobot commented 9 years ago

For our purpose, big JSON data are those whose object model are too big to fit the available memory. Currently, most of JSONP API's (e.g. JsonArray) requires that whole JSON object model be in memory. The only available API for processing big JSON seems to be JsonParser, but that is too low level to be useful.

Since Java stream operations is a useful tool to use for querying, filtering and processing JSON data, we should explore ways for constructing streams from JsonParse events. Of course the objects in the stream should be high level enough to be useful, like elements of an array or object. Streams should also be implemented in such a way that only the elements that take part in the stream operations need to be in memory. For instance, those elements that are filtered out can be discarded.

Not sure at this stage if it's best to add API's to JsonParser to create stremas, or we'll need new classes/interfaces that works with JsonParser. In any case, we probably need to add some features in JsonParser (e.g. skip to the end of an array) that will be useful when processing big JSON.

Affected Versions

[1.1]

glassfishrobot commented 9 years ago

Reported by kchung

glassfishrobot commented 7 years ago

@m0mus said: It was fixed by adding skip* methods to JsonParser.

glassfishrobot commented 9 years ago

Issue-Links: blocks JSON_PROCESSING_SPEC-66

glassfishrobot commented 9 years ago

Was assigned to kchung

glassfishrobot commented 7 years ago

This issue was imported from java.net JIRA JSON_PROCESSING_SPEC-70

glassfishrobot commented 7 years ago

Marked as fixed on Wednesday, March 8th 2017, 3:23:34 am