issue with u2eve

[kernelschmelze]

u2eve did not recognise the last event created by snort. u2spewfoo show the last event, so it's successfully updated to the unified2 log file. u2eve only reports the prior event.

[jasonish]

Can you tell me how you are running u2eve? The idstools source distribution contains a file, tests/merged.log that contains 6 events. u2spewfoo prints 6 event records along with some packet records. u2eve also prints 6 records. I admit thats a rather simplistic test, but I need more information in order to replicate.

[kernelschmelze]

Snort created a new spool file with only one entry. u2eve did not recognise this. I've triggered a test command (nmap) and snort created the second event. This time u2eve reports the prior event, but not the last (nmap) event. It does no matter if u2eve run's in follow mode or as single command. I've made a copy of the snort spool file and can provide it, if you need it. u2eve run's with --snort-conf /etc/snort/snort.conf --directory /var/log/snort --prefix snort.u2 --output /var/log/snort/eve.json --bookmark --follow

[jasonish]

Yes, please provide. u2eve may present some delay due to the formatting of unified2 files. It will read an event record, then wait for a following packet record before outputting the event. This is so it can present the alert/event and the packet in a single record. If there is no following packet, the event won't be output until the next event. But I have test cases for this.

Does your unified output contain packets?

[kernelschmelze]

Yes, the unified log contain also packets. Meanwhile i've made a other test and removed the bookmark and u2eve output file. This time u2eve reports all events. I think it is a problem with the bookmark thing. I forgot to backup the bookmark file, so i think it is senseless to provide the snort file without the bookmark file.

[kernelschmelze]

I've switched back to follow mode with bookmark file, and the u2eve output file looks like malformed. The last event appear two times and the last entry miss some information (event-type and following until packets).I used the u2eve output file as input for evebox. Any plan for that evebox can directly read unified files? debug.gz .

[jasonish]

Thanks. I'm still not sure why this is happening, but will look into it further but it may be a couple days.

Any plan for that evebox can directly read unified files?

I've been thinking about this recently. As I already have unified2 and rule parsers written in Go it wouldn't actually take much time. Perhaps I'll consider it more seriously, but won't be able to act on it for at least a week.

[jasonish]

Any chance you can share your unified2 file? Privately if needed - ish@unx.ca

[jasonish]

Oh, and what does your "output unified2" look like in your snort.conf? I may have made some assumptions that don't fit all use cases.

[kernelschmelze]

Snort run with the default settings. output unified2: filename snort.u2, limit 128

[jasonish]

I've pushed an update to git master that may help out your situation. While I was unable to replicate corrupt JSON, I was able to create incomplete records, which led to an erroneous record after restart. So I'm curious if it would help your use case as well.

[jasonish]

Any update here? I'd like to tag a release soon and would like to mark this as fixed.

Thanks.

[kernelschmelze]

Sorry, busy week. So far it looks good. I did not see any malformed json entries and currently all snort events are reported correct. I'm suprised about the --follow command. u2eve now always run in continuous mode.

[jasonish]

I'm suprised about the --follow command. u2eve now always run in continuous mode.

Oops, fixing.

[jasonish]

See 8200f4cbbe1e2757e761f2da78abb913e4fd3d38

Closing.