ingest custom log sources with custom index names

[AtulSinghMothi]

Hello Team, Our application is deployed on multiple VMs and application is generating multiple log files. UFs are installed on all VMs and log1 is indexed with index=log1,log2 is indexed with index=log2 etc.

Now, VMs are going to be replaced by kubernetes and it is expected that log generation is going to remain same. Could you please advise how can I assign index=log1 for log1,index=log2 for log2 etc.

Thank you!

[hvaghani221]

You can add splunk.com/index pod annotation. SCK will automatically send logs to index defined in splunk.com/index.

You can refer doc for detailed information.

[Kiyoshi-Miyake]

Hi harshit-splunk, Your sugestion is helpful me too. thanks! but, i tried to set these annotation, but, these are overwridden by the following parts.

      # extract pod_uid and container_name for CRIO runtime
      {{- if eq .Values.containers.logFormatType "cri" }}
      <filter tail.containers.var.log.pods.**>
        @type jq_transformer
        jq '.record | . + (.source | capture("/var/log/pods/(?<pod_uid>[^/]+)/(?<container_name>[^/]+)/(?<container_retry>[0-9]+).log")) | .sourcetype = ("{{ .Values.sourcetypePrefix }}:container:" + .container_name) | .splunk_index = {{ or .Values.global.splunk.hec.indexName .Values.splunk.hec.indexName | default "main" | quote }}'
      </filter>
      {{- end }}

templates/configMap.yaml line 308...

Do I need to remove the matching of regex for these sourcetype, index from the line?

Thanks!

[hvaghani221]

Hey @Kiyoshi-Miyake, that is actually a bug. It's a good catch. I would love if you raise a PR to fix this :)

[Kiyoshi-Miyake]

Hi @harshit-splunk, I created the PR. Can you check it.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.