nginx logs not parsing

[TRADe-Lynx]

We use fluent-bit helm chart version version 0.47.7 on Kubernetes(EKS) out to OpenSearch viewing with Kibana trying to get the Nginx logs separate into field like so:

but instead all fields are inside one "log" field makes it very hard to search and visualize NOT what we need.

like so:

Our nginx log format is default

log_format simple '$remote_addr - $remote_user [$time_local] '
                '"$request" $status $body_bytes_sent '
                '"$http_referer" "$http_user_agent"';

so dose the nginx parser

config:
  service: |
    [SERVICE]
        Daemon Off
        Flush {{ .Values.flush }}
        Log_Level {{ .Values.logLevel }}
        Parsers_File /fluent-bit/etc/parsers.conf
        Parsers_File /fluent-bit/etc/conf/custom_parsers.conf
        HTTP_Server On
        HTTP_Listen 0.0.0.0
        HTTP_Port {{ .Values.metricsPort }}
        Health_Check On
  ## https://docs.fluentbit.io/manual/pipeline/inputs
  inputs: |
    [INPUT]
        Name tail
        Path /var/log/containers/*.log
        multiline.parser docker, cri
        Tag kube.*
        Mem_Buf_Limit 5MB
        Skip_Long_Lines Off
        Refresh_Interval 10

    [INPUT]
        Name tail
        Tag  nginx.*
        Path /var/log/containers/nginx*.log
        Parser nginx

  ## https://docs.fluentbit.io/manual/pipeline/filters
  filters: |
    [FILTER]
        Name kubernetes
        Match kube.*
        Merge_Log On
        K8S-Logging.Parser On
        Merge_Log_Key log_processed
        K8S-Logging.Exclude On
    [FILTER]
        Name parser
        Match nginx.*
        Key_Name log
        Parser nginx

  ## https://docs.fluentbit.io/manual/pipeline/outputs
  outputs: |
    [OUTPUT]
        Name es
        Match *
        Host vpc-test-opensearch-test.es.amazonaws.com
        Port 443
        TLS On
        Logstash_Format On
        Logstash_Prefix test-fluent
        Retry_Limit False
        Suppress_Type_Name On
        Generate_ID On
        Type flb_type
        Time_Key @timestamp
        Replace_Dots On

  customParsers: |
    [PARSER]
        Name nginx
        Format regex
        Regex  ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")
        Time_Key time
        Time_Format %d/%b/%Y:%H:%M:%S %z

• we also tried with Nginx pod annotations fluentbit.io/parser: nginx which didn't work.
• we also minimize the nginx log to one element and one regex which didn't work?
• we also had match with regex tester https://regex101.com/r/oPEqMk/1 but it still didn't work Any idea how to get it working?

[edsiper]

please provide your full fluent bit log file and make sure the parser is recognized

[lynomet]

please provide your full fluent bit log file and make sure the parser is recognized

fluent-bit.log

[lynomet]

@edsiper Any idea?