search

splunk / eventgen

Splunk Event Generator: Eventgen
Apache License 2.0
380 stars 179 forks source link

[BUG] Multiple strptime formats in token replacement #369

Open jmeixensperger opened 4 years ago

jmeixensperger commented 4 years ago

Describe the bug Events are not generated when I use multiple strptime expressions for a single token.

To Reproduce Generate with included files

Expected behavior Both strptime expressions are evaluated/written with the correct replaytimestamp

Actual behavior Nothing is generated

Sample files and eventgen.conf file eventgen.conf:

[sample.mobilemusic.csv]
sampletype = csv
outputMode = stdout
end = 1
mode = replay

token.0.token = ((\w+\s+\d+\s+\d{2}:\d{2}:\d{2}:\d{3})|(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}:\d{3}))
token.0.replacementType = replaytimestamp
token.0.replacement = ["%b %d %H:%M:%S:%f", "%Y-%m-%d %H:%M:%S:%f"]

sample:

index,host,source,sourcetype,_raw
eventgenTest,splunk,/var/log/radius.log,radius,May 27 18:28:11:000 aaa2 radiusd[12676]:[ID 959576 local1.info] INFO RADOP(13) acct start for 5559031692@splunktel.com 10.94.63.34 from 130.253.37.97 recorded OK.
eventgenTest,splunk,/var/log/httpd/access_log,access_custom,"2012-05-27 18:28:11:112 10.2.1.35 POST /playhistory/uploadhistory - 80 - 10.94.63.34 ""Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3"" 503 0 0 468 1488"
eventgenTest,splunk,/var/log/httpd/access_log,access_custom,"2012-05-27 18:28:11:125 10.2.1.35 GET /sync/addtolibrary/01011207201000005652000000000047 - 80 - 10.94.63.34 ""Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3"" 200 0 0 468 1488"
eventgenTest,splunk,/var/log/httpd/access_log,access_custom,"2012-05-27 18:28:11:137 10.2.1.35 GET /sync/addtolibrary/01011207201000005652000000000047 - 80 - 10.94.63.34 ""Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/7534.48.3"" 503 0 0 468 1488"
eventgenTest,splunk,/var/log/radius.log,radius,May 27 18:28:11:199 aaa2 radiusd[12676]:[ID 959576 local1.info] INFO RADOP(13) acct stop for 5559031692@splunktel.com 10.94.63.34 from 130.253.37.97 recorde

Do you run eventgen with SA-eventgen? No

If you are using eventgen with pip module mode (please complete the following information):

Additional context Relevant token.X.replacement docs on passing a list of strptime expressions (seems supported): "For ["list","of","strptime"], only used with replaytimestamp, a JSON formatted list of strptime formats to try."

jmeixensperger commented 4 years ago

On second thought, I don't think this is the right/accepted approach, nor required by anyone right now--Closing This conf actually came from our own tutorial4. It's not being used anywhere, but there is also a small mention of this feature in our docs. We should investigate the issue and add tests/docs for this feature if we plan on supporting it.