OOM errors for building giant maps of attribute values by primary key, in memory, where there is no current use case for lookups.
When the reproduceable OOM happens, the attribute value map is holding ~915M of memory across ~2.6 million k/v pairs. The individual values in this case are comma separated lists of IDs.
Stream the attribute value query results instead of collecting them into an intermediary map.
Additional Notes
Going by code searches performed by Steve, Dave, and I, it appears that the AbstractAttributeReporter#getAttributeValue method is only called by WordCloudAttributeReporter, and that reporter disregards the keys and simply iterates through the values.
Problem
OOM errors for building giant maps of attribute values by primary key, in memory, where there is no current use case for lookups.
When the reproduceable OOM happens, the attribute value map is holding ~915M of memory across ~2.6 million k/v pairs. The individual values in this case are comma separated lists of IDs.
Example Value:
Proposed Solution
Stream the attribute value query results instead of collecting them into an intermediary map.
Additional Notes
Going by code searches performed by Steve, Dave, and I, it appears that the
AbstractAttributeReporter#getAttributeValuemethod is only called byWordCloudAttributeReporter, and that reporter disregards the keys and simply iterates through the values.