OOMs due to object creations in ADMDataParser

GoogleCodeExporter commented 8 years ago

Pouria, Keren, and I ran into memory issues (OOMs) when loading data into 
AsterixDB.
I did some profiling and eventually found the issue is in ADMDataParser.

The following change doesn't recycle object pools before (or after) parsing 
each ADM record such that the ListObjectPools are ever growing:
https://github.com/apache/incubator-asterixdb/commit/c66d23a5ac65ec5218ee47134ae
a423fd62a32cc

I just created a fix for review: https://asterix-gerrit.ics.uci.edu/#/c/334/.

Original issue reported on code.google.com by buyingyi@gmail.com on 30 Jul 2015 at 4:12

GoogleCodeExporter commented 8 years ago

Original comment by buyingyi@gmail.com on 30 Jul 2015 at 4:17

Added labels: Priority-Critical
Removed labels: Priority-Medium

GoogleCodeExporter commented 8 years ago

Original comment by buyingyi@gmail.com on 30 Jul 2015 at 4:30

Changed state: Fixed

simlaudato / asterixdb

OOMs due to object creations in ADMDataParser #912