NASAWorldWind / WorldWindServerKit

The NASA WorldWind Server Kit (WWSK) is an open source Java project that assembles GeoServer for easy distribution and implementation.
92 stars 54 forks source link

Investigate Export GeoPackage memory usage #104

Closed emxsys closed 7 years ago

emxsys commented 7 years ago

Investigate what appears to be excess memory consumption when exporting a GeoPackage via WPS.

emxsys commented 7 years ago

The large memory usage appears to be the result of internal caching by GeoServer. Heap dumps show an accumulation of byte[] arrays that are referenced by SoftReference objects. They appear to be released in occasional major garbage collections.

I compared two versions of the GeoPackage export routines, one using additional WebMap.dispose and RasterCleaner.addImage calls and the other without. Heap dumps from both were relatively identical with regards to the number and size of the byte[] objects.

emxsys commented 7 years ago

However, the number of surviving generation continues to climb during successive GeoPackage exports:

image

emxsys commented 7 years ago

Comparing heap dumps taken before and after a large GeoPackage export shows an accumulation of SoftReference entries:

image

Although the SoftReference referent values are eventually set to NULL, the queue size continues to grow and the overhead of the GC appears to rise.

emxsys commented 7 years ago

The org.geoserver.jai.ConcurrentTileFactory may be the owner of the SoftReference nodes.

TODO: Investigate GeoServer Image Processing and Raster Access settings.

emxsys commented 7 years ago

I disabled the logic used in issue #4 (CustomRenderedImageMapOutputFormat and MapResponseOutputStreamAdaptor) to see if the intermediate BufferedImage had an impact, but the results were the same.

emxsys commented 7 years ago

I'm ceasing the investigation for now to pursue higher priority tasks.