olafhartong / sysmon-modular

A repository of sysmon configuration modules
MIT License
2.66k stars 589 forks source link

(Updated) Workflows, priority sorting for rules, rule improvements #171

Closed cnnrshd closed 1 year ago

cnnrshd commented 1 year ago

Note - duplicate of (closed) #170 , tracking a patch branch instead of master.

This is a large PR that addresses three key things:

  1. Automatically generating an ATT&CK Matrix
  2. Adding support for priority-based rule sorting
  3. Better organization and deduplication of many rules, specifically focused on LOLBAS

Matrix

Addition

  1. Added a workflow to build an attack matrix on Push or PR (Same logic as generate-config, copy-pasted the structure). This matrix has scores associated with each technique, and I just use jq to replace the value of the techniques array.

Changes

  1. Changed the configuration generation from running on Windows to Ubuntu. This dropped the build time from ~ 2-3 minutes to ~20 seconds. Running builds on Windows doubles the used time (ex. a 40 second build on Linux counts as one minute, while on Windows it counts as 80 seconds -> 2 minutes)
  2. Updated Actions (checkout and upload-artifact) to version 3 - from my understanding, this updated the Node version. Updated so GitHub stopped warning me on builds

Priority-based Rules Sorting

I noticed this issue when going through the sysmonconfig.xml - there's a PowerShell chunk that will overrule many CommandLine rules below this Rule ([like here](https://github.com/olafhartong/sysmon-modular/blob/965652e8b333174705c2c13ad200e4c3810ae0e5/sysmonconfig.xml#

Additions

  1. Simple Python script that can merge based on a similar format to preexisting Include Lists - the only difference is it takes a CSV with two columns, filepath and priority
  2. A config formatted using this script
  3. A simple template
  4. Schemaversion is dynamic and based on the highest schema version of provided rules.

Changes

  1. Configs generated using this script maintain comments and proper XML indentation is enforced, increasing readability and allowing easier cross-referencing of rule files

General modifications

Most of my effort has been on the lolbas - network and process creation.

  1. Reduced duplicate MTIRE ATT&CK techniques being reported - rules that have multiple filters in an AND have been reduced to one tag, since it is one detection.
  2. Removed many duplicate rules
  3. Fixed inaccurate technique ids
olafhartong commented 1 year ago

Oh wow thanks for this big (well documented) push, I'll go through it asap

olafhartong commented 1 year ago

awesome additions. really like the python approach, I will look into making all pipelines, input files and templates compatible there too.

The pwsh file is becoming a small mess, like that it works on any windows box but for the pipelines and maintenance this is way cleaner. Thanks for the huge work!

cnnrshd commented 1 year ago

i agree - powershell is really convenient, but there's some good utility from using python (libraries, far more people can write it, very good xml support). glad I could help!