Sending logs from specific pod only

[ALNBAND]

Hi,

Is there a way to send logs from only one pod? e.g. fluent is reading all cluster/nodes pod logs, if there is a way to configure it to send to splunk only one pod logs ?

[chaitanyaphalak]

You can achieve that using annotations, checkout https://github.com/splunk/splunk-connect-for-kubernetes#managing-sck-log-ingestion-by-using-annotations

[ALNBAND]

1. So by default it doesn't send any data? I need explicitly configure from which pods to send the data? or there is option to disable all and one by one explicitly ?
2. Does the tool have buffering on a disk, e.g. if Splunk is down to prevent a data loss ?

[hvaghani221]

1. So by default it doesn't send any data? I need explicitly configure from which pods to send the data? or there is option to disable all and one by one explicitly ?

By default it will send logs from all the container. You can exclude log from any container by adding splunk.com/exclude annotation.

2. Does the tool have buffering on a disk, e.g. if Splunk is down to prevent a data loss ?

Yes, it'll buffer data in memory. Here you can modify the configuration from here (https://github.com/splunk/splunk-connect-for-kubernetes/blob/develop/helm-chart/splunk-connect-for-kubernetes/values.yaml#L418-L427)

[hvaghani221]

You can also add customFilter to only allow from only specific pod. For example,

splunk-kubernetes-logging:
  customFilters:
    includeOnlyFilter:
      tag: tail.containers.**
      type: grep
      body: |
        <regexp>
          key pod
          pattern /^etcd-.*$/
        </regexp>

Reference: https://github.com/splunk/splunk-connect-for-kubernetes/blob/develop/helm-chart/splunk-connect-for-kubernetes/values.yaml#L471-L506

[ALNBAND]

@harshit-splunk - thanks, ok the customFilter looks promising, can I disable all the pods logs with simple config? and then add the fiter

[hvaghani221]

@ALNBAND, there is no direct way to disable log collection from all pods unless you modify template file. By applying previous filter, fluend will capture and drop all other logs except from etcd pod

[hvaghani221]

@ALNBAND should I close the issue?

[matthewmodestino]

you can also set the path value to focus in on specifc regex or comma separated values:

https://github.com/splunk/splunk-connect-for-kubernetes/blob/4b460c24c13bfa8f3d240bcbd29cfb6e25be1076/helm-chart/splunk-connect-for-kubernetes/values.yaml#L79

By default we scoop up everything. With a bit of thought you could scope the path we watch down to just a specific namespace or pod by examining the path to the pod on the node and adjusting the path value as necessary....k8s puts pod_namespace_container-name-container-id in the path. So you can use that to scope down to just what you want.

https://docs.fluentd.org/input/tail#path

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.