Open cazter opened 3 years ago
+1. This is something we are also looking at.
+1
Late update to this post but we filtered by making the following changes to the configmap:
Allowing regex, the below line excludes each of the defined namespaces; "pg*, monitoring and devops.
Exclude_Path /var/log/containers/*_pg*_*.log, /var/log/containers/*_monitoring_*.log, /var/log/containers/*_devops_*.log
apiVersion: v1
data:
filter-kubernetes.conf: |
[FILTER]
Name record_modifier
Match *
Record cluster_name ${CLUSTER_NAME}
[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc:443
K8S-Logging.Exclude ${K8S_LOGGING_EXCLUDE}
fluent-bit.conf: |
[SERVICE]
Flush 1
Log_Level ${LOG_LEVEL}
Daemon off
Parsers_File parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
@INCLUDE input-kubernetes.conf
@INCLUDE output-newrelic.conf
@INCLUDE filter-kubernetes.conf
input-kubernetes.conf: |
[INPUT]
Name tail
Tag kube.*
Path ${PATH}
Exclude_Path /var/log/containers/*_pg*_*.log, /var/log/containers/*_monitoring_*.log, /var/log/containers/*_devops_*.log
Parser ${LOG_PARSER}
DB /var/log/flb_kube.db
Mem_Buf_Limit 7MB
Skip_Long_Lines On
Refresh_Interval 10
output-newrelic.conf: |
[OUTPUT]
Name newrelic
Match *
licenseKey ${LICENSE_KEY}
endpoint ${ENDPOINT}
parsers.conf: |
# Relevant parsers retrieved from: https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
[PARSER]
Name cri
Format regex
Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: newrelic-logging
meta.helm.sh/release-namespace: devops
labels:
app: newrelic-logging
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: newrelic-logging
chart: newrelic-logging-1.4.4
heritage: Helm
release: newrelic-logging
name: newrelic-logging-fluent-bit-config
namespace: devops
Trying to reduce our NR ingest volume, specifically the logs portion. While I see that there is support for removing pods using an annotation it would be much easier/cleaner if we could specify namespaces to enable or disable logging.