[newrelic-logging] filter by namespace

[cazter]

Trying to reduce our NR ingest volume, specifically the logs portion. While I see that there is support for removing pods using an annotation it would be much easier/cleaner if we could specify namespaces to enable or disable logging.

[jkalwar]

+1. This is something we are also looking at.

[ghost]

+1

[cazter]

Late update to this post but we filtered by making the following changes to the configmap:

Allowing regex, the below line excludes each of the defined namespaces; "pg*, monitoring and devops.

Exclude_Path      /var/log/containers/*_pg*_*.log, /var/log/containers/*_monitoring_*.log, /var/log/containers/*_devops_*.log

apiVersion: v1
data:
  filter-kubernetes.conf: |
    [FILTER]
        Name           record_modifier
        Match          *
        Record         cluster_name ${CLUSTER_NAME}

    [FILTER]
        Name           kubernetes
        Match          kube.*
        Kube_URL       https://kubernetes.default.svc:443
        K8S-Logging.Exclude ${K8S_LOGGING_EXCLUDE}
  fluent-bit.conf: |
    [SERVICE]
        Flush         1
        Log_Level     ${LOG_LEVEL}
        Daemon        off
        Parsers_File  parsers.conf
        HTTP_Server   On
        HTTP_Listen   0.0.0.0
        HTTP_Port     2020

    @INCLUDE input-kubernetes.conf
    @INCLUDE output-newrelic.conf
    @INCLUDE filter-kubernetes.conf
  input-kubernetes.conf: |
    [INPUT]
        Name              tail
        Tag               kube.*
        Path              ${PATH}
        Exclude_Path      /var/log/containers/*_pg*_*.log, /var/log/containers/*_monitoring_*.log, /var/log/containers/*_devops_*.log
        Parser            ${LOG_PARSER}
        DB                /var/log/flb_kube.db
        Mem_Buf_Limit     7MB
        Skip_Long_Lines   On
        Refresh_Interval  10
  output-newrelic.conf: |
    [OUTPUT]
        Name  newrelic
        Match *
        licenseKey ${LICENSE_KEY}
        endpoint ${ENDPOINT}
  parsers.conf: |
    # Relevant parsers retrieved from: https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf
    [PARSER]
        Name         docker
        Format       json
        Time_Key     time
        Time_Format  %Y-%m-%dT%H:%M:%S.%L
        Time_Keep    On

    [PARSER]
        Name cri
        Format regex
        Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$
        Time_Key    time
        Time_Format %Y-%m-%dT%H:%M:%S.%L%z
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: newrelic-logging
    meta.helm.sh/release-namespace: devops
  labels:
    app: newrelic-logging
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: newrelic-logging
    chart: newrelic-logging-1.4.4
    heritage: Helm
    release: newrelic-logging
  name: newrelic-logging-fluent-bit-config
  namespace: devops