Open DaveCTurner opened 1 year ago
Pinging @elastic/es-search (Team:Search)
The issue probably comes from building global ordinals where we account for the memory usage but never break:
This line of code looks the same since 2014 so we have never circuit break here.
Can confirm that this is still a problem on 7.17.18, and it seems on larger text aggregations, fielddata cache grows uncontrollably, ignoring all default breaker limits (heap is 32gb):
Even though setting indices.fielddata.cache.size
solves these issues, it does not seem like a valid solution :pray:
Pinging @elastic/es-analytical-engine (Team:Analytics)
A v8.6 user on the forums reported experiencing OOMEs and when they analysed the heap dump they found that a high fraction of their 3GiB heap was used by the fielddata cache.
GET /_nodes/_all/stats/breaker?filter_path=nodes.*.breakers.fielddata
agrees:They have worked around this problem by setting
indices.fielddata.cache.size: 1gb
but I think it's a bug for the fielddata cache to grow without bounds by default like this.