search

wagga40 / Zircolite

A standalone SIGMA-based detection tool for EVTX, Auditd and Sysmon for Linux logs
671 stars 91 forks source link

Ingest XML events #52

Closed Blaapje closed 1 year ago

Blaapje commented 1 year ago

How much work would it take to ingest xml data rather than evtx files?

The XML data is generated by: https://github.com/williballenthin/EVTXtract. The tools recovers (partial) xml records from corrupted partitions.

Could you indicate if this is feasible and give some pointers towards ingesting xml data? I'd be happy to submit a PR if I get this to work.

wagga40 commented 1 year ago

As a quick response I can say that Zircolite already handles XML formatted logs because it is the native format of sysmon for linux logs (--sysmon4linux). I haven’t tested with logs coming from EVTXtract.

wagga40 commented 1 year ago

Ok forgot what I've said, I took a look to the output of EVTXtract and it is a little bit more difficult than that...

Blaapje commented 1 year ago

It's rather interesting that both EVTXtract and evtx_dump are both maintained by @williballenthin. I will see if I can get this working.

Edit: I don't know how I came to this conclusion but this is obviously not the case.

wagga40 commented 1 year ago

In Zircolite everything (EVTX, Auditd, Sysmon For Linux) is converted to JSON lines format. These operations are handled by the evtxExtractor class. For each log file the Logs2JSON function is called for each line in the log file to convert it to JSON.

Since the XML output of EVTXtract is not in the "1 event per line" format, the first thing is to handle this. The easiest way to to it would be to modify EVTXtract (check here) (but I guess that's not what you want to do) and use XML to JSON conversion logic like the one used here.

If you have samples I can help...

Blaapje commented 1 year ago

I've made a small sample available here: https://gist.github.com/Blaapje/23e378748db737960a90c723c2dc8676

wagga40 commented 1 year ago

Thanks, I will try this week end.

wagga40 commented 1 year ago

Hi @Blaapje, worked on it ! I currently only handle events beginning with <event> and not the other ones (i.e <record>) since they don’t contains the fields that allows to apply sigma on them. I will push something today.

wagga40 commented 1 year ago

You can try with this branch : https://github.com/wagga40/Zircolite/tree/evtxtract. Please provide some feedback.