fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.59k stars 1.53k forks source link

opentelemtry input error: invalid value type in pack_any_value #7840

Open Frapschen opened 11 months ago

Frapschen commented 11 months ago

Bug Report

Describe the bug

I use opentelemetry collector to watch Kube events, using the k8sobject receiver, and sent them to fluentbit. opentelemetry collector config:

exporters:
  logging:
    verbosity: normal
    sampling_initial: 1
    sampling_thereafter: 1000
  otlphttp/event:
    endpoint: http://insight-agent-fluent-bit:4318
    compression: gzip
extensions:
  health_check: {}
  memory_ballast:
    size_in_percentage: 40
processors:
  batch: {}
  memory_limiter:
    limit_percentage: 80
    spike_limit_percentage: 25
    check_interval: 1s
receivers:
  k8sobjects:
    auth_type: serviceAccount
    objects:
      - name: events
        mode: watch
        group: events.k8s.io/v1
service:
  telemetry:
    logs:
      level: "info"
    metrics:
      address: 0.0.0.0:8888
  extensions:
    - health_check
    - memory_ballast
  pipelines:
    logs:
      exporters:
        - logging
        - otlphttp/event
      processors:
        - memory_limiter
        - batch
      receivers:
        - k8sobjects

fluentbit config:

[SERVICE]
    Daemon              Off
    Flush               5
    Log_Level           debug
    Parsers_File        parsers.conf
    HTTP_Server         On
    HTTP_Listen         0.0.0.0
    HTTP_Port           2020
    Health_Check        On
[INPUT]
    name opentelemetry
    listen 0.0.0.0
    tag_key otlp
    port 4318
[OUTPUT]
    name stdout
    match *

fluentbit stdout log:

Fluent Bit v2.1.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/08/18 16:32:39] [ info] [fluent bit] version=2.1.8, commit=1d83649441, pid=1
[2023/08/18 16:32:39] [ info] [storage] ver=1.4.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/08/18 16:32:39] [ info] [cmetrics] version=0.6.3
[2023/08/18 16:32:39] [ info] [ctraces ] version=0.3.1
[2023/08/18 16:32:39] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2023/08/18 16:32:39] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2023/08/18 16:32:39] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4318
[2023/08/18 16:32:39] [ info] [output:stdout:stdout.0] worker #0 started
[2023/08/18 16:32:39] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2023/08/18 16:32:39] [ info] [sp] stream processor started
[2023/08/18 16:33:12] [error] [otel]: invalid value type in pack_any_value
[2023/08/18 16:33:12] [error] [otel] Failed to convert log record body
[2023/08/18 16:33:12] [error] [otel] marshalling error
[2023/08/18 16:33:19] [error] [otel]: invalid value type in pack_any_value
[2023/08/18 16:33:19] [error] [otel] Failed to convert log record body
[2023/08/18 16:33:19] [error] [otel] marshalling error

Expected behavior

Screenshots

image

Your Environment

pandres-varian commented 8 months ago

OTEL protocol specify that if AnyValue cannot be transformed it should be considered empty

opentelemetry/proto/common/v1/common.proto

Fluentbit just discards the value otlp_pack_any_value

github-actions[bot] commented 5 months ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] commented 5 months ago

This issue was closed because it has been stalled for 5 days with no activity.