[Integration]Service Metrics Dynamic-Dashboard Generator

[YANG-DB]

What is the integration resource ?

We would like to auto-generate a dashboard for each service so that the SRE would have a simple and easy way of viewing and monitoring the system from the service perspectives .

What is the integration source protocol ? The OTEL services (via traces) and metrics signals will be part of this visualization

What is the integration target protocol ? Data-Prepper's otel-metrics-* index containing the metrics info - from OTEL collector Data-Prepper's otel-v1-apm-service-map index containing the services relations info - from OTEL collector

Which agents would you use to ship this data ?

• Data prepper exporter
• Jaeger exporter

Which Dashboards would you be using ?

This is a vega-visualization component that is both configurable and agile for updates without any code changes

---

Do you have any additional context?

The next query is a candidate for data to be shown

POST otel-metrics-2024.01.31/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "kind.keyword": "SUM"
          }
        }
      ]
    }
  }, 
  "aggs": {
    "service_name": {
      "terms": {
        "field": "serviceName.keyword",
        "size": 10
      },
      "aggs": {
        "metrics_name": {
          "terms": {
            "field": "name.keyword",
            "size": 10
          },
          "aggs": {
            "metrics_type": {
              "terms": {
                "field": "kind.keyword",
                "size": 10
              }
            }
          }
        }
      }
    }
  }
}