fluent / fluent-operator

Operate Fluent Bit and Fluentd in the Kubernetes way - Previously known as FluentBit Operator
Apache License 2.0
578 stars 246 forks source link

Allow grep filter logical_op #1316

Open ba-work opened 1 month ago

ba-work commented 1 month ago

Is your feature request related to a problem? Please describe.

The grep filter support is incomplete, I want to be able to at least use multiple excludes: https://docs.fluentbit.io/manual/pipeline/filters/grep#multiple-conditions

Describe the solution you'd like

Ideally I could do something like this to filter namespaces at a cluster level:

apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterFilter
metadata:
  name: namespaces
  labels:
    fluentbit.fluent.io/component: logging
    fluentbit.fluent.io/enabled: 'true'
spec:
  filters:
    - grep:
       # currently you can only do 1 exclude and you can't specify a logical_op
        logicalOp: or
        exclude: $kubernetes['namespace_name'] default
        exclude: $kubernetes['namespace_name'] kube-system

Additional context

The current workaround is multiple grep filters like this:

apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterFilter
metadata:
  name: namespaces
  labels:
    fluentbit.fluent.io/component: logging
    fluentbit.fluent.io/enabled: 'true'
spec:
  filters:
    - grep:
        exclude: $kubernetes['namespace_name'] default
    - grep:
        exclude: $kubernetes['namespace_name'] kube-system

I'm not sure what the real impact of this discrepancy is.