search

jfrog / log-analytics

JFrog Log Analytics
Apache License 2.0
18 stars 13 forks source link

Feature Request: set host and sourcetype in Splunk events #10

Closed danbarr closed 3 years ago

danbarr commented 4 years ago

For Splunk, it would be helpful if events included the original host, and a unique sourcetype to help search for events. Otherwise, all events are indexed by Splunk with "host" equal to the HEC's host:port, and with a sourcetype of just "httpevent".

This could be handled a few ways with Fluentd, but in my config I just added the following two lines to the final <match jfrog.**> section in the td-agent.conf file (made from the fluent.conf.rt or fluent.conf.xray files in this repo):

sourcetype_key log_source
default_host "#{hostname}"

This sets sourcetype to the same value as your log_source field, and host to the originating server's hostname. By setting sourcetype, the JFrog Platform Log Analytics app for Splunk could (and should) be improved to use sourcetype=jfrog. instead of just in its dashboard searches. In a large Splunk environment, simply searching for * causes those dashboards to take forever and/or time out.

peters95 commented 3 years ago

Thanks for the feedback we have added this to the backlog to be implemented to address the concerns for large Splunk environments.

peters95 commented 3 years ago

This has been resolved as part of the app v1.1.0 updates. Please download the latest fluent conf and splunkbase app to verify.