Support for `Provider_name` and `Data[x]` notation to the field mapping

Yamato-Security / hayabusa

Hayabusa (隼) is a sigma-based threat hunting and fast forensics timeline generator for Windows event logs.

GNU General Public License v3.0

2.17k stars 189 forks source link

Support for `Provider_name` and `Data[x]` notation to the field mapping #1350

Closed fukusuket closed 3 months ago

fukusuket commented 3 months ago

Currently, the field mapping feature does not support following point:

Provider_name matching
- e.g. Since the Application log has multiple Provider_names, we need to convert the field value only when the Provider_name matches.(in addition to Channel, EventID)
Data[x] notation filed mapping

We want to support following rule/field conversion, so I'll implement above point.

https://github.com/Yamato-Security/hayabusa-rules/pull/666

fukusuket commented 3 months ago

Specification memo:

If possible, make Provider_Name optional
If possible, allow multiple values to be set for Provider_Name(It is preferable to have one yaml file)
Do not output the value before field value conversion to ExtraFieldInfo (csv/json)

fukusuket commented 3 months ago

Do not output the value before field value conversion to ExtraFieldInfo (csv/json)

I have looked into the feasibility of implementation and it does not appear to be easy to achieve 🤔 The main reasons are the following two points:

Under the Data field is an array (not simple key value)
The exclusion process for Details and ExtraFieldInfo is a bit complicated
- e.g. https://github.com/Yamato-Security/hayabusa/pull/1146
- This is a different issue, but I think the fix will be as complicated as this one.

@YamatoSecurity I think I'm going to create the issue of output to ExtraFieldInfo as a separate issue and hold off on implementation, what do you think?

YamatoSecurity commented 3 months ago

@fukusuket I see, sure, we can hold off on the implementation for now. I'm thinking it might be better to save the Data fields as different fields instead of in an array for JSON. For example: Data-1, Data-2, etc... This way it will be easier to deal with when importing records into elastic stack, etc...

fukusuket commented 3 months ago

@YamatoSecurity

Do not output the value before field value conversion to ExtraFieldInfo (csv/json)

Sorry many times, I looked into it more and found how to implement this just now(though my current implementation is even more complicated...😇). I'll create PR.

I see, It would be nice if eliminating Array would make it easier to integrate with things like ElasticSearch :)