grafana / tempo

Grafana Tempo is a high volume, minimal dependency distributed tracing backend.
https://grafana.com/oss/tempo/
GNU Affero General Public License v3.0
4.03k stars 522 forks source link

DOCS: Add more detail to Tempo troubleshooting section #1135

Open mellieA opened 2 years ago

mellieA commented 2 years ago

Provide more detail in this section: https://grafana.com/docs/tempo/latest/troubleshooting/agent/#metrics to describe where the metrics can be viewed.

The agent publishes a few Prometheus metrics that are useful to determine how much trace traffic it is receiving and successfully forwarding. These are a good place to start when diagnosing tracing Agent issues.

traces_receiver_accepted_spans traces_receiver_refused_spans traces_exporter_sent_spans traces_exporter_send_failed_spans

Suggest adding:

The metrics are visible at http://localhost:12345/metrics if you are using the Grafana Agent to send traces to Grafana Cloud. In your Grafana Cloud instance, they can be checked in the grafanacloud-usage datasource:

Screen Shot 2021-11-24 at 12 06 07 PM
GrafanaWriter commented 2 years ago

@KMiller-Grafana - FYI. Triaging to backlog.

knylander-grafana commented 2 years ago

These updates could go https://grafana.com/docs/tempo/latest/grafana-agent/

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity. Please apply keepalive label to exempt this Issue.

knylander-grafana commented 1 month ago

Looking at this issue. The information about the metrics usage for grafanacloud-usage can be added to the Cloud Traces documentation and to the troubleshooting section for Grafana Alloy. The first sentence is still correct, according to the Alloy docs. https://grafana.com/docs/alloy/latest/troubleshoot/controller_metrics/

knylander-grafana commented 1 month ago

PR - https://github.com/grafana/tempo/pull/4137