sonatype-nexus-community / nexus-repository-composer

Composer support for the Nexus Repository Manager (work in progress!)
Eclipse Public License 1.0
210 stars 82 forks source link

Make ComposerJsonProcessor more memory-efficient #4

Open fjmilens3 opened 6 years ago

fjmilens3 commented 6 years ago

The ComposerJsonProcessor class performs multiple operations on JSON:

These are currently done by unmarshaling all the contents of the JSON files into memory as maps. For the time being this seems to work just fine, but a more parsimonious way to do this would be to parse the JSON using something like the JsonParser API rather than the ObjectMapper API that we're currently using.

(Sonatype employees can reference our internal NPM implementation for ideas here. In rare situations for NPM large JSON file could exhaust available memory when we used the ObjectMapper-based approach, which eventually convinced us to switch to the JsonParser for these kinds of operations.)

fjmilens3 commented 6 years ago

Some of this will change depending on the outcome of other work in progress, so it would be wise to check in before picking this one up. However, this may be something that someone with familiarity or interest in Jackson, Composer JSON formats, or JSON processing in general might be able to pick up without a larger understanding of the entire project.