wagga40 / Zircolite

A standalone SIGMA-based detection tool for EVTX, Auditd and Sysmon for Linux logs
671 stars 91 forks source link

Unable to create table #81

Closed luckman212 closed 2 months ago

luckman212 commented 2 months ago

Nice looking project!

I'm trying to run it on an M1 Mac Mini running macOS 14.6 + Python 3.12 + a virtualenv.

I cloned the repo, created a new venv, and installed requirements.full.txt without incident.

I pointed it at a directory of .evtx files recently exported from a Windows 11 PC. The directory contains 516 files totalling 336MB.

But, it's failing during the run after Creating model:

image

Any ideas on how to debug this?

wagga40 commented 2 months ago

Hi,

Any ideas on how to debug this?

to debug, there is a --debug option and you can check (and may be share here) the output of zircolite.log.

Since you have a lot of different EVTX files, it is possible that this is a "Too many columns" error linked to a SQLite default limitation. Check this previous issue for details.

If it is that, a simple solution would be to split the number of files handled by one instance of Zircolite. For example, you create two directories with half of the EVTX in each one and run two Zircolite in parallel.

luckman212 commented 2 months ago

Thank you. I ran it again with --debug, here is the logfile:

zircolite.log

But, I did as you suggest, split the dump of evtx files into 3 separate dirs and was able to successfully process them.

wagga40 commented 2 months ago

Thank you.

Looking at your logs, it was indeed a "Too many columns" error :

2024-08-03 13:20:02 DEBUG    zircolite:823 executeQuery    [-] too many columns on logs

I will had something in the docs and may be a check to warn users.

PS : if you often work with a lot of logs be sure to check this section of the docs to speed up your analysis.

Thanks for sharing.