open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.92k stars 2.28k forks source link

StatsD receiver is not handling `count` attribute correctly for Prometheus Remote Write #10897

Closed datsabk closed 1 year ago

datsabk commented 2 years ago

The purpose and use-cases of the new component

We are trying to utilise Open Telemetry Collector for moving StatsD metrics to Prometheus. We tested and found that all the other metrics type except count is not going through well. It ends up with an issue as below:

2022-06-10T21:51:15.055+0530    error   exporterhelper/queued_retry.go:183  Exporting failed. The error is not retryable. Dropping data.    {"kind": "exporter", "name": "prometheusremotewrite", "error": "Permanent error: invalid temporality and type combination", "dropped_items": 111}
go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send
    go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/queued_retry.go:183
go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send
    go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/metrics.go:132
go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1
    go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/queued_retry_inmemory.go:118
go.opentelemetry.io/collector/exporter/exporterhelper/internal.consumerFunc.consume
    go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/internal/bounded_memory_queue.go:82
go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func2
    go.opentelemetry.io/collector@v0.51.0/exporter/exporterhelper/internal/bounded_memory_queue.go:69

Example configuration for the component

receivers:
  statsd:
    endpoint: "0.0.0.0:8125" # default
    aggregation_interval: 10s  # default
    enable_metric_type: false   # default
    is_monotonic_counter: true # default
    timer_histogram_mapping:
      - statsd_type: "histogram"
        observer_type: "gauge"
      - statsd_type: "timing"
        observer_type: "gauge"
exporters:
  # file:
  #   path: ./test.json
  prometheusremotewrite:
    endpoint: <my-aws-amp-url>
    auth:
      authenticator: sigv4auth
    remote_write_queue:
      enabled: true
      queue_size: 5000
      num_consumers: 1
extensions:
  sigv4auth:
    region: "us-west-2"
    assume_role: 
      session_name: "otel_session"
service:
  extensions: [sigv4auth]
  pipelines:
    metrics:
      receivers: [statsd]
      exporters: [prometheusremotewrite]

Telemetry data types supported

Metrics - count datatype not supported

Is this a vendor-specific component? If so, are you proposing to contribute this as a representative of the vendor?

Sponsor (Optional)

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

jmacd commented 1 year ago

I think this may have been fixed by changes in the PRW exporter since June.

awiddersheim commented 1 year ago

@jmacd Thats awesome! Any chance you have a link to the PR that fixes it?

awiddersheim commented 1 year ago

There are a few other issues that seem related:

https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/5187 https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/5250 https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/4909 https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/11434

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 year ago

This issue has been closed as inactive because it has been stale for 120 days with no activity.