search

outflanknl / RedELK

Red Team's SIEM - tool for Red Teams used for tracking and alarming about Blue Team activities as well as better usability in long term operations.
BSD 3-Clause "New" or "Revised" License
2.35k stars 371 forks source link

VT quota management #287

Open MarcOverIP opened 1 year ago

MarcOverIP commented 1 year ago

Im getting Virus Total quota issues although it seems that I still have space left in my quota. Output from daemon.log reads:

2022-12-05 12:26:26,399 - DEBUG - alarm_filehash - module.py - check_hashes -- Results from VirusTotal: {'02bc35ae711eee0977c0cc099954ecad': {'result': 'skipped, quota reached'},
<here follows a very long list of hashes>

My hypothesis is that although the quota says 240 left for now, the actual list of items to check exceeds 240. I checked this manually for one call and indeed it wanted to check 386 hashes in one go. VT will see the too big result come in and therefor nothing will be checked. This is an endless loop as the amount of to be checked items will only grow.

It would be ideal if alarm_filehash module would actually parse the output of VT's report on available quota, and tune the amount of to be checked hashed accordingly.

fastlorenzo commented 1 year ago

This has normally the same behavior has HA module, it first checks the remaining quota:

https://github.com/outflanknl/RedELK/blob/5f50fc6b047d65bb1cd845b6993efc66b3077580/elkserver/docker/redelk-base/redelkinstalldata/scripts/modules/alarm_filehash/ioc_vt.py#L27-L63

Then checks each hash one by one until the quota is reached, the remainder should be done on next run when the quota is available again:

https://github.com/outflanknl/RedELK/blob/5f50fc6b047d65bb1cd845b6993efc66b3077580/elkserver/docker/redelk-base/redelkinstalldata/scripts/modules/alarm_filehash/ioc_vt.py#L91-L100

It's normally not a "all or nothing", it should be a smart queue :smile: