different output with json parser

[Frapschen]

Bug Report

Describe the bug there is a different resoult when add the Parsers_File parsers.conf in [SERVICE]. like this configuration demo:

[SERVICE]
          Daemon Off
          Flush  5
          Log_Level debug
          Parsers_File parsers.conf
          HTTP_Server On
          HTTP_Listen 0.0.0.0
          Health_Check On

[INPUT]
    Name              tail
    Tag               audit.*
    Path              /var/log/*/audit/*.log
    Parser            json
    Read_from_Head    true
    Refresh_Interval  10
    Mem_Buf_Limit     5MB

[OUTPUT]
    Name stdout
    Match *

To Reproduce if add the Parsers_File parsers.conf in [SERVICE], the stdout will output :

fluent-bit_1      | [100] audit.var.log.kubernetes.audit.kube-apiserver-audit.log: [1656663991.091225212, {"kind"=>"Event", "apiVersion"=>"audit.k8s.io/v1", "level"=>"Metadata", "auditID"=>"509f911b-f654-46ce-bbae-a762513ca8b4", "stage"=>"ResponseComplete", "requestURI"=>"/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/k8s-master1?timeout=10s", "verb"=>"update", "user"=>{"username"=>"system:node:k8s-master1", "groups"=>["system:nodes", "system:authenticated"]}, "sourceIPs"=>["127.0.0.1"], "userAgent"=>"kubelet/v1.21.6 (linux/amd64) kubernetes/d921bc6", "objectRef"=>{"resource"=>"leases", "namespace"=>"kube-node-lease", "name"=>"k8s-master1", "uid"=>"1c96197e-54b9-488d-a4bd-a4ce05dac7ca", "apiGroup"=>"coordination.k8s.io", "apiVersion"=>"v1", "resourceVersion"=>"54815710"}, "responseStatus"=>{"metadata"=>{}, "code"=>200}, "requestReceivedTimestamp"=>"2022-05-27T09:20:56.582244Z", "stageTimestamp"=>"2022-05-27T09:20:56.601405Z", "annotations"=>{"authorization.k8s.io/decision"=>"allow", "authorization.k8s.io/reason"=>""}}]

if delete the Parsers_File parsers.conf in [SERVICE], the stdout will output :

fluent-bit_1      | [13] audit.var.log.kubernetes.audit.kube-apiserver-audit.log: [1656664131.089326555, {"log"=>"{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"572c8c3d-625d-46dc-83d8-9aeebb274bdd","stage":"ResponseComplete","requestURI":"/apis/crd.projectcalico.org/v1/felixconfigurations?limit=500","verb":"list","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["10.6.182.104"],"userAgent":"argocd-application-controller/v0.0.0 (linux/amd64) kubernetes/$Format","objectRef":{"resource":"felixconfigurations","apiGroup":"crd.projectcalico.org","apiVersion":"v1"},"responseStatus":{"metadata":{},"status":"Failure","reason":"ServiceUnavailable","code":503},"requestReceivedTimestamp":"2022-05-27T09:20:56.454755Z","stageTimestamp":"2022-05-27T09:20:56.455811Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":""}}"}]

• Steps to reproduce the problem:

Expected behavior I expected the stdout will output the same formated log whatever add or delete the Parsers_File parsers.conf in [SERVICE]

Your Environment

• Version used: fluent-bit:1.9.3
• Environment name and version : docker 20.10.14
• Operating System and version: macos 12.3.1 M1

[nokute78]

I think this is not a bug since json parser is defined at parsers.conf. https://github.com/fluent/fluent-bit/blob/v1.9.6/conf/parsers.conf#L36

If Parsers_File is not added, fluent-bit can't find the definition and in_tail doesn't parse. https://docs.fluentbit.io/manual/pipeline/parsers/configuring-parser

[Frapschen]

I think this is not a bug since json parser is defined at parsers.conf. https://github.com/fluent/fluent-bit/blob/v1.9.6/conf/parsers.conf#L36

If Parsers_File is not added, fluent-bit can't find the definition and in_tail doesn't parse. https://docs.fluentbit.io/manual/pipeline/parsers/configuring-parser

ok, thinks