search

open-telemetry / opentelemetry-demo

This repository contains the OpenTelemetry Astronomy Shop, a microservice-based distributed system intended to illustrate the implementation of OpenTelemetry in a near real-world environment.
https://opentelemetry.io/docs/demo/
Apache License 2.0
1.54k stars 873 forks source link

Spanmetrics panel always returns no results #1623

Open flands opened 2 weeks ago

flands commented 2 weeks ago

Bug Report

Which version of the demo you are using? 1.10.0

Symptom

Spanmetrics panel always returns no results

What is the expected behavior?

Spanmetrics panel returns results - this was the case, at least, in version 1.7.0

What is the actual behavior?

Spanmetrics panel always returns no results

Reproduce

Download version 1.10.0 and run make start. Once started, open Grafana (http://localhost:8080/grafana) and navigate to Dashboards > Demo > Demo Dashboard. Look at the panels under Spanmetrics. Switch to any other service and the same behavior is experienced. If you scroll down, you will see most of the other panel work (some do not have logs today so those also return no results but for a different reason).

Additional Context

This appears to be a regression. A workaround is to manually edit each widget and for the query letter (A, B, C, etc) option settings, manually setting the Min step to something like 30s.

puckpuck commented 1 week ago

Spanmetrics data takes several minutes to show up. The data is aggregated on minute intervals, and because the charts are all rate charts you also need multiple data points before anything can render for them.

How long did you wait after starting the demo before visiting the dashboard?

flands commented 6 days ago

More than an hour and I tried every service - see attachments.

Screenshot 2024-06-30 at 10 04 50 AM Screenshot 2024-06-30 at 10 04 58 AM