[META] Multi-tenancy support in OpenSearch

[ansjcy]

Please describe the end goal of this project

Introduce and enable multi-tenancy as a first class citizen in OpenSearch by associating traffic with specific users or workloads. This will allow better categorization, management, troubleshooting and analysis of subsets of traffic hitting an OpenSearch cluster.

We can start from the search use cases and build a generic framework/solution, which can be extend it to other workflow like indexing.

Supporting References

Related RFC

• [ ] https://github.com/opensearch-project/OpenSearch/issues/13341

Potential downstream consumers

• [ ] https://github.com/opensearch-project/OpenSearch/issues/11429
• [ ] https://github.com/opensearch-project/OpenSearch/issues/11173
• [ ] https://github.com/opensearch-project/OpenSearch/issues/10938
• [ ] https://github.com/opensearch-project/OpenSearch/issues/12084
• [ ] Slow logs, we haven't created an issue as mentioned in the RFC, but if we add the labels in the source field slow log will automagically have the labels.

Issues/PRs

MVP version:

• [x] POC: https://github.com/opensearch-project/OpenSearch/pull/13374
• [ ] Add the ability to attach tenancy labels in a SearchRequest, allowing clients to provide tenancy information in search requests.
• [ ] Develop a rule-based labeling system where a component on the cluster assigns labels to incoming search requests.

Related component

Search

[andrross]

[Triage - attendees 1 2 3 4] @ansjcy Thanks for filing. Looking forward to seeing progress here.

[kaushalmahi12]

Hi @ansjcy thanks for opening this issue. I have an open PR for this for QSB use case(search workloads): https://github.com/opensearch-project/OpenSearch/pull/13713