Add alerting and enable anomaly-detection over benchmark metrics

[rishabh6788]

Is your feature request related to a problem? Please describe

Currently we are not tracking any regression over the nightly benchmark run metrics being generated across multiple versions and features.

Describe the solution you'd like

Add alerting and enable anomaly-detection over performance metrics. Any threshold breach should notify the team using email or any paging solution.

Describe alternatives you've considered

No response

Additional context

No response

[rishabh6788]

Appropriate alarm and monitoring has been set up over overall indexing and latency metrics.

[gaiksaya]

Closing this issue based on previous comment. Please feel free to re-open in case more action items are remaining. Thanks!

[rishabh6788]

Reopening to track adding monitoring and alerting at each operation level metrics.

[dblock]

[Catch All Triage - 1, 2, 3]