search

grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
1.04k stars 111 forks source link

[Tempo] Improve vision on dropped spans #555

Open joe-elliott opened 2 years ago

joe-elliott commented 2 years ago

Currently observability on dropped spans is poor. The agent is based on the OTEL collector which has this issue:

https://github.com/open-telemetry/opentelemetry-collector/issues/2434

Additionally if batches are dropped b/c the queue is full they go unmetriced. This is logged, but not metriced as a failed send: "Dropping data because sending_queue is full"

https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/monitoring.md#secondary-monitoring

Let's keep an eye on these issues and revendor when they are fixed. If we have spare cycles these are also good issues for upstream contributions.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.