Yamato-Security / hayabusa

Hayabusa (隼) is a sigma-based threat hunting and fast forensics timeline generator for Windows event logs.
GNU General Public License v3.0
2.22k stars 198 forks source link

[bug] aggregation condition rule count does not show up in `Events with hits` #1375

Closed fukusuket closed 2 months ago

fukusuket commented 3 months ago

Describe the bug aggregation condition rule count does not show up in Events with hits(and Top 5 computers) It's probably the similar cause as #1373, but I'll create a separate issue to make it easier to understand.

Step to Reproduce ./hayabusa-2.16.0-mac-aarch64 csv-timeline -d ../hayabusa-sample-evtx -r rules/hayabusa/builtin/Security/LogonLogoff/Logon/Sec_4625_Med_LogonFail_WrongPW_PW-Guessing_Cnt.yml -w

ref: Sec_4625_Med_LogonFail_WrongPW_PW-Guessing_Cnt.yml

Actuail behavior

...
Timestamp · RuleTitle · Level · Computer · Channel · EventID · RecordID · Details · ExtraFieldInfo
2016-09-20 01:50:06.513 +09:00 · PW Guessing · med · - · - · - · - · Count: 3558 ¦ IpAddress: 192.168.198.149 · -
...
Results Summary:

Events with hits / Total events: 0 / 26,341 (Data reduction: 26,341 events (100.00%))

Total | Unique detections: 1 | 1
Total | Unique critical detections: 0 (0.00%) | 0 (0.00%)
Total | Unique high detections: 0 (0.00%) | 0 (0.00%)
Total | Unique medium detections: 1 (100.00%) | 1 (0.00%)
Total | Unique low detections: 0 (0.00%) | 0 (100.00%)
Total | Unique informational detections: 0 (0.00%) | 0 (0.00%)

Dates with most total detections:
critical: n/a, high: n/a, medium: 2016-09-20 (1), low: n/a, informational: n/a

Top 5 computers with most unique detections:
critical: n/a
high: n/a
medium: n/a
low: n/a
informational: n/a

Expected behavior

...
Timestamp · RuleTitle · Level · Computer · Channel · EventID · RecordID · Details · ExtraFieldInfo
2016-09-20 01:50:06.513 +09:00 · PW Guessing · med · - · - · - · - · Count: 3558 ¦ IpAddress: 192.168.198.149 · -
...
Results Summary:

Events with hits / Total events: 1 / 26,341 (Data reduction: 26,341 events (100.00%))

Total | Unique detections: 1 | 1
Total | Unique critical detections: 0 (0.00%) | 0 (0.00%)
Total | Unique high detections: 0 (0.00%) | 0 (0.00%)
Total | Unique medium detections: 1 (100.00%) | 1 (0.00%)
Total | Unique low detections: 0 (0.00%) | 0 (100.00%)
Total | Unique informational detections: 0 (0.00%) | 0 (0.00%)

Dates with most total detections:
critical: n/a, high: n/a, medium: 2016-09-20 (1), low: n/a, informational: n/a

Top 5 computers with most unique detections:
critical: n/a
high: n/a
medium: somehostname
low: n/a
informational: n/a

Environment

Additional context This seems to be because the aggregation condition rule is not counted in the line below. https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/afterfact.rs#L500-L502

fukusuket commented 3 months ago

Current impl memo:

Related Struct

detection.rs#L44-L51 https://github.com/Yamato-Security/hayabusa/blob/95ee553a8abe9b0f3886ed60673452e2496cd06c/src/detections/detection.rs#L44-L51

message.rs#L32-L45

https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/detections/message.rs#L32-L45

count.rs#L221-L226

https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/detections/rule/count.rs#L221-L233

mod.rs#L30-L35 https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/detections/rule/mod.rs#L30-L35

mod.rs#L375-L386 https://github.com/Yamato-Security/hayabusa/blob/95ee553a8abe9b0f3886ed60673452e2496cd06c/src/detections/rule/mod.rs#L375-L386

Related Sequence

count up

rule/mod.rs#L85-L104 https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/detections/rule/mod.rs#L85-L110

rule/count.rs#L20-L74 https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/detections/rule/count.rs#L20-L74

main.rs#L1576 https://github.com/Yamato-Security/hayabusa/blob/95ee553a8abe9b0f3886ed60673452e2496cd06c/src/main.rs#L1576

output rule count

detection.rs#L763

https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/detections/detection.rs#L763-L776

afterfact.rs#L500-L502

https://github.com/Yamato-Security/hayabusa/blob/efaca57e93c6e298a3f2afcd9dbbad93e545dff8/src/afterfact.rs#L500-L502

fukusuket commented 3 months ago

New impl memo

 #[derive(Debug, Clone, PartialEq, Eq, Default)] 
/// countなどのaggregationの結果を出力する構造体
pub struct AggResult {
    /// countなどの値
    pub data: i64,
    /// count byで指定された条件のレコード内での値
    pub key: String,
    /// countの括弧内指定された項目の検知されたレコード内での値の配列。括弧内で指定がなかった場合は長さ0の配列となる
    pub field_values: Vec<String>,
    ///検知したブロックの最初のレコードの時間とEventID
    pub id_time_pair : Vec<(String, DateTime<Utc>)>
}
 #[derive(Debug, Clone, PartialEq, Eq, Default)] 
 pub struct DetectInfo { 
     pub detected_time: DateTime<Utc>, 
     pub rulepath: CompactString, 
     pub ruleid: CompactString, 
     pub ruletitle: CompactString, 
     pub level: CompactString, 
     pub computername: CompactString, 
     pub eventid: CompactString, 
     pub detail: CompactString, 
     pub ext_field: Vec<(CompactString, Profile)>, 
     pub agg_result: Option<Aggresult>, 
     pub details_convert_map: HashMap<CompactString, Vec<CompactString>>, 
 }