search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.35k stars 24.55k forks source link

Remove unnecessary boxed Double usage in SortingDigest #109439

Open original-brownbear opened 2 months ago

original-brownbear commented 2 months ago

We've run into a heap dump where org.elasticsearch.tdigest.SortingDigest used up GBs of heap. We can cut the heap use of the thing by 75% by just moving to unboxed double and double[]. Currently, org.elasticsearch.tdigest.SortingDigest#byteSize pretends that this is the case, counting 8b per element when in fact it's at least 32b (8 for the reference and 24 for each Double). Also sorting Double vs sorting a double[] array is going to be much much slower as well.

elasticsearchmachine commented 2 months ago

Pinging @elastic/es-analytical-engine (Team:Analytics)