[Docs] - Elasticsearch - Split into user and administrator sections

quiltdata / quilt

Quilt is a data mesh for connecting people with actionable data

https://quiltdata.com

Apache License 2.0

1.33k stars 91 forks source link

[Docs] - Elasticsearch - Split into user and administrator sections #4201

Open QuiltSimon opened 2 days ago

QuiltSimon commented 2 days ago

Currently the documentation page on Elasticsearch has sections relating to end user functionality (querying) and administrator functions (monitoring, auto-tuning and configuration).

Suggest we split this into two sections to better fit into each of the sections Quilt Platform User and Quilt Platform Administrator.

drernie commented 2 days ago

While we are at it: we should probably pull out the Catalog-related entries from the (quilt3) FAQ

Similarly, the Catalog 'Advanced' could be merged into the main "Search & Query"

drernie commented 2 days ago

Possibly irrelevant, but the Admin API docs should be in Advanced (and need to be more readable).

drernie commented 2 days ago

Suggests four types:

Tutorial: I am new to the tool, and I want a hands-on onboarding
How-to: I know about the tool, but I wonder how to perform a specific task
Explanation: I want to learn about architecture choices, advanced concepts, etc.
Reference: I want to get an exhaustive list of all features/methods

drernie commented 8 hours ago

NOTE: Warn people that re-indexing > 1M objects can get expensive.

Alternative:

Opensearch out of memory
Overview stats incorrect / not updating Check SNS topic and/or S3 event source (chain is S3 events => SNS => SQS => Lambda => ES ES is out of disk Workaround if dropped data while problem was occurring but is now fixed: re-index (not recommended for more than ~10M object buckets unless you're ready for a long re-index)