search

fluent / fluentd-kubernetes-daemonset

Fluentd daemonset for Kubernetes and it Docker image
Apache License 2.0
1.26k stars 978 forks source link

Unable to exclude ELB-HealthChecker/2.0 logs #1387

Closed raunoodem closed 1 year ago

raunoodem commented 1 year ago

Image used - fluent/fluentd-kubernetes-daemonset:v1.15.1-debian-cloudwatch-1.0

I'm trying to minimize the logs sent to CloudWatch and trying to not send kube-probe and load balancer healthchecks to cloudwatch.

Currently using exclude filter like this - 

      <filter **>
        @type grep
        <exclude>
          key log
          pattern /GET \/health/
        </exclude>
      </filter>

Before Appling this filter I could see the following log lines - 10.128.54.48 - - [12/Sep/2022:12:35:41 +0000] "GET /health HTTP/1.1" 200 15 "-" "kube-probe/1.23+" "-" 10.128.48.97 - - [12/Sep/2022:12:35:42 +0000] "GET /health HTTP/1.1" 200 15 "-" "ELB-HealthChecker/2.0" "-"

After applying the above mentioned filter the kube-probe/1.23+ healtcheck is no longer sent to CloudWatch. However the ELB healtcheck logs are still sent to cloudwatch.

raunoodem commented 1 year ago

Found the issue, Needed to move the filter before the parser.