Missing Events - Githubissues

sophos / Sophos-Central-SIEM-Integration

Simple integration script for 3rd party systems such as SIEMs. Offers command line, file or syslog output in CEF, JSON or key-value pair formats.

121 stars 70 forks source link

Missing Events #53

Open gautaus opened 3 years ago

gautaus commented 3 years ago

Hi,

Lately, we have been noticing Sophos central events lost (not being pulled) by the v1 of the integration script. I guess it has something to do with the state file not properly being read by the script. We have deployed v2 of the script too and have seen similar reports. Is anyone else facing similar issues.

Cheers, Gautham

ramksophos commented 3 years ago

Hi @gautaus, do you have any events visible in Sophos Central in the 24 hours preceding your script run? If there are no events in that period, the script will fetch nothing. It would be good if you can indicate the command line parameters you are using, any logs output by the script as well as the state file (both suitably redacted), and examples of events you are seeing in Central but not in the script output.

gautaus commented 3 years ago

Hi @rkamat,

We did a count on the Sophos central events over last 24 hours and got the following difference.

Events from script - 128 events Events in CSV exported from Sophos Central - 376 events Difference - 248 events

Cheers, Gautham

nandishdoshi-sophos commented 3 years ago

Hi @gautaus , can you pls let us know from which screen did we exported Events csv? Also did we check if missing 248 events were acknowledged/resolved

sahoo8920 commented 3 years ago

Hi @nandishdoshi-sophos , these events were exported from Sophos Central Dashboard. Does the script only pull acknowledged or resolved events?

VatsalJagani commented 2 years ago

I noticed the same issue of missing data. I can see the events on the host as well as on sophos-central dashboard but not in the events collected by this script.

I think the most likely scenario is that the events are coming out of order on sophos-central. And when Sophos API responds next_cursor it returns the timestamp of the latest event but some events with lower timestamp come to Sophos later.
I'm not an expert in this but to test the above theory I've created the following code change, I'm not sure if this is going to work or not.

VatsalJagani commented 2 years ago

I've submitted the below PR to resolve the issue - https://github.com/sophos/Sophos-Central-SIEM-Integration/pull/69

Those who are facing the issue, please give the code change a try and provide your feedback.