search

slackhq / astra

Astra is a structured log search and analytics engine developed by Slack and Salesforce
https://slackhq.github.io/astra/
MIT License
214 stars 30 forks source link

Add multitenancy support to Astra #223

Open bryanlb opened 2 years ago

bryanlb commented 2 years ago

We need to introduce the ability to support multitenancy through index names. The way this is roughly expected to happen is certain indexers in a cluster will be responsible for dedicated indexes, which will be determined by the preprocessor. When an indexer creates a snapshot, it will mark which indexes are contained within that snapshot. As a query is received snapshots will be filtered to only return those matching a snapshot.

If multiple indexes, wildcard, or glob-based index names are received the query layer will resolve these to exact index matches when forwarding to index/cache layer.

pre-processor > indexer(s)

         index-foo (indexers 1-10)
         index-bar (indexer 11)
         index-baz (indexer 11)

grafana > query > cache/indexer

         Query will look at inbound requested index(s), then filter to 
           snapshots that are known to have that data. Cache/indexer
           will also then filter to the exact requested indexes.
github-actions[bot] commented 2 months ago

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 30 days.