We've run into a heap dump where org.elasticsearch.tdigest.SortingDigest used up GBs of heap.
We can cut the heap use of the thing by 75% by just moving to unboxed double and double[].
Currently, org.elasticsearch.tdigest.SortingDigest#byteSize pretends that this is the case, counting 8b per element
when in fact it's at least 32b (8 for the reference and 24 for each Double).
Also sorting Double vs sorting a double[] array is going to be much much slower as well.
We've run into a heap dump where
org.elasticsearch.tdigest.SortingDigest
used up GBs of heap. We can cut the heap use of the thing by 75% by just moving to unboxeddouble
anddouble[]
. Currently,org.elasticsearch.tdigest.SortingDigest#byteSize
pretends that this is the case, counting 8b per element when in fact it's at least 32b (8 for the reference and 24 for eachDouble
). Also sortingDouble
vs sorting adouble[]
array is going to be much much slower as well.