aodn / geoserver-build

Configures a GeoServer war containing extensions and configuration required by AODN instances
Other
2 stars 4 forks source link

JavaDuck excessive memory usage #241

Closed jonescc closed 7 years ago

jonescc commented 7 years ago

Javaduck currently reads all time varying variables into memory (uncompressed) before appending them to the output file. For collections such as CARS_monthly this can consume a large amount of memory inside and outside the java heap (over 670 MiB).

It is not necessary to read all data into memory before writing it out for this dataset. The dataset can be processed one time slice at a time.