opensearch-project / dashboards-observability

Visualize and explore your logs, traces and metrics data in OpenSearch Dashboards
https://opensearch.org/docs/latest/observability-plugin/index/
Apache License 2.0
16 stars 56 forks source link

[BUG] Observability Metrics page : only HISTOGRAM metrics are displayed #2236

Open glelarge opened 1 month ago

glelarge commented 1 month ago

This issue follows #1927 and #2172 to display metrics in the Observability Metrics page.

What is the bug? In the Observability Metrics page, displayed metrics are filtered to HISTOGRAM. Perhaps not a bug but user is not alerted that page is filtered to histograms.

How can one reproduce the bug? Steps to reproduce the behavior:

  1. Go to the Observability Metrics page
  2. Select OpenTelemetry as Metrics source
  3. Select the **ss4o_metrics-* index as Otel Index**
  4. Only histogram metrics are displayed :

Screenshot from 2024-10-29 16-41-57

Debug Looking at the code to find the origin of the 500 error of #2172 , I saw this query filtered to kind HISTOGRAM.

What is the expected behavior? All metrics should be displayed. At least, something should indicate that only histogram metrics are displayed.

I'm quite new to the vast telemetry world, so I'm not sure to clearly understand all the ideas, philosophies, dependencies and so on, so why other kind of metrics (GAUGE, COUNTERS...) are not displayed ? Is it clearly intended for the Observability Metrics page, or would it be added in a future release ? Would it ever make sense to add other metrics kinds ? To go further in the understanding, does it mean the software under monitoring must provide its own histograms ?

What is your host/environment? Deployed with Docker images OpenSearch Dashboards 2.17.1 OpenSearch Data Prepper 2.10.1