fluent / fluent-operator

Operate Fluent Bit and Fluentd in the Kubernetes way - Previously known as FluentBit Operator
Apache License 2.0
587 stars 250 forks source link

bug: Can't set `net.dns.resolver` #1297

Closed dex4er closed 3 months ago

dex4er commented 3 months ago

Describe the issue

I can't set net.dns.resolver LEGACY and without this setting the opensearch output doesn't work at all. Currently this bug makes me fluent-operator not usable at all.

To Reproduce

My output:

apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterOutput
metadata:
  name: opensearch
  labels:
    fluentbit.fluent.io/enabled: "true"
    fluentbit.fluent.io/mode: fluentbit-only
spec:
  match: "*"
  opensearch:
    host: redacted
    port: 443
    bufferSize: 10M
    type: _doc
    logstashFormat: true
    logstashPrefix: redacted
    logstashDateFormat: "%Y%m%d"
    timeKey: "@timestamp"
    timeKeyFormat: "%Y-%m-%dT%H:%M:%S"
    timeKeyNanos: false
    includeTagKey: true
    tagKey: log_name
    generateID: false
    writeOperation: create
    replaceDots: true
    traceOutput: false
    traceError: true
    currentTimeIndex: false
    suppressTypeName: true
    networking:
      DNSResolver: LEGACY

The config file from ConfigMap doesn't have any DNS entry:

[Output]
    Name    opensearch
    Match    *
    Host    redacted
    Port    443
    Buffer_Size    10M
    Type    _doc
    Logstash_Format    true
    Logstash_Prefix    redacted
    Logstash_DateFormat    %Y%m%d
    Time_Key    @timestamp
    Time_Key_Format    %Y-%m-%dT%H:%M:%S
    Time_Key_Nanos    false
    Include_Tag_Key    true
    Tag_Key    log_name
    Generate_ID    false
    Write_Operation    create
    Replace_Dots    true
    Trace_Output    false
    Trace_Error    true
    Current_Time_Index    false
    Suppress_Type_Name    true

Anyway: the current code is broken and would generate net.dns.prefer_ipv4 instead of net.dns.resolver:

https://github.com/fluent/fluent-operator/blob/master/apis/fluentbit/v1alpha2/plugins/net_types.go#L53

Expected behavior

I should see

net.dns.resolver LEGACY

in the ConfigMap

Your Environment

EKS 1.30 with:

- Fluent Operator version: 3.0.0
- Container Runtime: containerd

How did you install fluent operator?

helm install

Additional context

Without this option set fluent-bit can't send anything to Opensearch

The log from fluent-bit is:

[2024/08/13 17:15:59] [ warn] [net] getaddrinfo(host='redacted', err=11): Could not contact DNS server

This setting (net.dns.resolver LEGACY) fixes it.

dex4er commented 3 months ago

An explanation why DNSResolver: LEGACY is needed: https://github.com/fluent/fluent-bit/issues/7105