Extend tests for Observability stack

[jsmolar]

Summary

Observability automation should include tracing, metrics, dashboards, and alerts, for both Authorino/Limitador standalone and Kuadrant. Currently, the test suite implements Authorino tracing (supports the usage of Jaeger and Tempo as tracing backends) and metrics.

Documentation

Observability docs: https://docs.kuadrant.io/kuadrant-operator/doc/observability/examples/

Test coverage

Authorino standalone

• [x] Metrics
• [x] Tracing

Kuadrant

• [ ] Metrics
• [ ] Tracing - Waiting for https://github.com/Kuadrant/kuadrant-operator/issues/731 PR ready https://github.com/Kuadrant/testsuite/pull/458
• [ ] Dashboards
• [ ] Alerts

Authorino

• [x] Metrics - Closed by https://github.com/Kuadrant/testsuite/pull/499
• [ ] Tracing - This will be tested in Kuadrant when a global way of config is completed

Limitador

• [x] Metrics - Close by https://github.com/Kuadrant/testsuite/pull/493
• [ ] Tracing - This will be tested in Kuadrant when a global way of config is completed

[david-martin]

• Investigate and implement new tests for dashboards using Grafana

@jsmolar I wonder if it would be sufficient as an initial test of dashboards to deploy them with kustomize (mounted via configmap volumes), then use the dashboard api to verify they exist in grafana after it is ready?

A similar approach could be used for the example alert rules with the prometheus http api.

That, at the very least, should verify they are valid and are imported ok.