search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.25k stars 24.86k forks source link

[docs] ILM 200 million documents limit, explanation #99412

Open philippkahr opened 1 year ago

philippkahr commented 1 year ago

Description

The following ILM document has a recommendation to aim for 200 million documents per shard as a maximum. We set this automatically in all of our ILM settings. https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html#shard-size-recommendation

It would make sense to update the docs and explain a bit, why 200 million documents is a "magic" number and what the implications are of going over it. Too large cache entries, slower search performance, ... impacts on TSDB because of sorting algorithms...

elasticsearchmachine commented 1 year ago

Pinging @elastic/es-docs (Team:Docs)

elasticsearchmachine commented 1 year ago

Pinging @elastic/es-data-management (Team:Data Management)