open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.01k stars 2.33k forks source link

[testbed] Misleading "Dropped Span Count" metric reported in results #35866

Open bacherfl opened 3 hours ago

bacherfl commented 3 hours ago

Component(s)

testbed

What happened?

Description

Currently, the testbed stores several metrics, such as CPU and memory consumption in a benchmarks.json file after a test has been finished. These metrics also include a Dropped Span Count - this is however misleading, as this is also reported for tests where e.g. only logs or metrics are sent to the collector. Since this metric calculated based on the data items sent by the datasender and the data items received by the receiver, a better name would be Dropped Data Item Count.

Steps to Reproduce

Execute a load test not related to spans, e.g. this one, and investigate the benchmarks.json file.

Expected Result

The resulting benchmarks.json should include a Dropped Data Item Count metric

Actual Result

The benchmarks.json file includes a Dropped Span Count metric which is misleading that case, e.g.:

[
  {
    "name": "cpu_percentage_avg",
    "unit": "%",
    "value": 1.4635678556630998,
    "extra": "TestMetric10kDPS/Carbon - Cpu Percentage"
  },
  {
    "name": "cpu_percentage_max",
    "unit": "%",
    "value": 1.9655741202201407,
    "extra": "TestMetric10kDPS/Carbon- Cpu Percentage"
  },
  {
    "name": "ram_mib_avg",
    "unit": "MiB",
    "value": 59,
    "extra": "TestMetric10kDPS/Carbon- RAM (MiB)"
  },
  {
    "name": "ram_mib_max",
    "unit": "MiB",
    "value": 103,
    "extra": "TestMetric10kDPS/Carbon - RAM (MiB)"
  },
  {
    "name": "dropped_span_count",
    "unit": "spans",
    "value": 0,
    "extra": "TestMetric10kDPS/Carbon - Dropped Span Count"
  }
]

Collector version

v0.111.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04") Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

github-actions[bot] commented 3 hours ago

Pinging code owners: