splunk / eventgen

Splunk Event Generator: Eventgen
Apache License 2.0
376 stars 180 forks source link

Use variables within csv files to reuse substituted timestamps in other events [FEATURE/IMPROVEMENT] #339

Open DieterSchmitz opened 4 years ago

DieterSchmitz commented 4 years ago

Is your feature request related to a problem? Please describe. No

Describe the solution you'd like To clarify what we need it is the best to give you an example: We (and some of our customers) use SA-EventGen and import CSV files we created on our own. The content of the CSV file looks like this (simplified): 2019-10-29 15:14:04.403,UserA,"2019-10-29TT14:14:04.403+0100",IndexName,Source,MetricA 2019-10-29 15:14:05.216,UserA,"2019-10-29TT14:14:04.403+0100",IndexName,Source,MetricB

The first line contains two timestamps. The first is the timestamp for the event, and the second timestamp is (in our case) the user's logon time. Both timestamps were replaced by SA-Eventgen which is fine. At the moment SA-Eventgen does the same with the second line. So the generated events looks like this: "2019-10-29 15:14:04.403,UserA,"2019-10-29TT14:14:04.403+0100",IndexName,Source,MetricA "2019-10-29 15:14:05.216,UserA,"2019-10-29TT14:14:05.216+0100",IndexName,Source,MetricB The difference is that all timestamps in a single row have the same value.

We want to reuse the second timestamp of the first event (2019-10-29TT14:14:04.403+0100) in the second event again. So SA-Eventgen should not replace the second timestamp of the second line with the current date/time but with the already replaced of the first event.

jmeixensperger commented 4 years ago

I don't see an easy way to accomplish this without requiring individual token replacement entries for each timestamp. I generally want to avoid that to keep the size of eventgen.conf files down.

One thing you could also try is to re-format one of the timestamps in each line and edit your token replacement so that it only picks up said timestamp. I'm not sure how feasible this is for you or how you're creating the files. If you need any help/clarification, let me know.