open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.11k stars 2.39k forks source link

influxdbexporte fails to convert empty tags #22850

Closed katarzyna-kulpa closed 1 year ago

katarzyna-kulpa commented 1 year ago

Component(s)

exporter/influxdb

What happened?

Description

I want to collect metrics from kubernetes cluster, I have k8s_cluster as receiver and influxdb as exporter. I have everything deployed in kind cluster. I cannot export metrics to influxdb as I'm getting the follwoing error: error exporterhelper/queued_retry.go:401 Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "data_type": "metrics", "name": "influxdb", "error": "Permanent error: failed to convert OTLP metric to line protocol: failed to write point for gauge: Permanent error: failed to encode point: encoding point 13: invalid tag value k8s.namespace.name=\"\"", "dropped_items": 220}

It seems that influxdb exporter cannot convert/skip empty tags.

Steps to Reproduce

Start collecting k8scluster metrcis on kind cluster and export them to influxdb

Expected Result

Empty tags are skipped

Actual Result

Empty tags are failed to being converted

Collector version

v0.75.0

Environment information

Environment

Ubuntu 20.04, kind cluster

OpenTelemetry Collector configuration

exporters:
        influxdb:
            bucket: default
            endpoint: http://cmo-influxdb:80
            metrics_schema: telegraf-prometheus-v1
            org: cmo
            retry_on_failure:
                enabled: true
                initial_interval: 3s
                max_elapsed_time: 10s
                max_interval: 10s
            sending_queue:
                enabled: false
                num_consumers: 3
                queue_size: 10
            timeout: 500ms
            token: xx
        logging:
            verbosity: detailed
    processors:
        batch: {}
        filter/metricFilter:
            metrics:
                exclude: {}
                include: {}
        memory_limiter:
            check_interval: 5s

            limit_mib: "4000"
            spike_limit_mib: "500"
    receivers:
        eemi:
            protocols:
                grpc: {}
                http:
                    cors:
                        allowed_origins:
                            - http://*
                            - https//*
        k8s_cluster:
            auth_type: serviceAccount
            distribution: kubernetes
        otlp:
            protocols:
                grpc: {}
                http:
                    cors:
                        allowed_origins:
                            - http://*
                            - https//*
        syslog:
            protocol: rfc3164
            tcp:
                listen_address: 0.0.0.0:54526
    service:
        pipelines:
            metrics/k8scluster-influxdb-subscription:
                exporters:
                    - influxdb
                processors:
                    - memory_limiter
                    - batch
                    - filter/metricFilter
                receivers:
                    - k8s_cluster

Log output

2023-05-28T20:28:43.751Z        error   exporterhelper/queued_retry.go:401      Exporting failed. The error is not retryable. Dropping data.    {"kind": "exporter", "data_type": "metrics", "name": "influxdb", "error": "Permanent error: failed to convert OTLP metric to line protocol: failed to write point for gauge: Permanent error: failed to encode point: encoding point 13: invalid tag value k8s.namespace.name=\"\"", "dropped_items": 220}

Additional context

No response

github-actions[bot] commented 1 year ago

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

crobert-1 commented 1 year ago

/label -needs-triage