[FEATURE] document performance tuning options

[rursprung]

Is your feature request related to a problem?

an out-of-the-box setup of OpenSearch might not perform very well depending on the use-case without proper tuning.

What solution would you like?

provide some guidelines on how to tune OpenSearch for various setups:

• generic setup (roughly equal amounts of ingest & search requests)
• search heavy setup
• ingest heavy setup
• other common setups i'm not aware off??

guidelines for operations (i.e. without any changes to the actually stored data / applications interacting with OpenSearch) should include:

• how many nodes (and of which type) to run
• how many shards to use per index
• how many CPUs & how much RAM to configure per node (and, as more and more are now running on things like Kubernetes: explain how much to set as requests and how much as limits - this isn't something which generally needs to be considered when running directly on an OS)
• what storage to use
• network considerations
• ...?

guidelines for developers (i.e. things which might result in changes to the indices, search queries, applications interacting with OpenSearch, etc.) should include:

• index design considerations
• query considerations
• considerations / best-practices when ingesting data
• using bulk operations
• etc.

What alternatives have you considered?

for now we can still just check the recommendations for Elasticsearch 🙃

Do you have any additional context?

see the documentation from Elasticsearch (it didn't cover all the points i listed here):

• Tune for search speed
• Tune for indexing speed
• Tune for disk usage
• How to size your shards

note: i'm not an expert in any of this, you'd probably best get some input on this from people running large-scale clusters for the different use-cases in production. i'm sure you might be able to find somebody at AWS who's doing that 😉

[JeffHuss]

@hdhalter I'm working on upgrade and installation docs - if this was assigned to me intentionally please ping me on Slack to help me understand the changes to priorities.

[hdhalter]

@JeffH-AWS - Yes, I assigned you to collaborate with Nate on it. We'll need your input on this, since you have in-depth knowledge on the topic. Priorities haven't changed. Please look at this when you can. Thanks!

[hdhalter]

@dblock - Do you know anyone that might be able to help collect some of this data and document it?