No netflow events found

[ngms17]

Hi,

Although i am logging netflow events on my index, evebox does not show any netflow events. How can i solve that?

[jasonish]

How are you logging these events to your index? Have you tried the "flow" logs and see if those works?

[ngms17]

I am logging the events through a separate eve.json file called "eve-netflows.json". My suricata is divided into 3 json files "eve-alerts.json", "eve-netflows.json" and "eve-events.json" (this one logs http, tls, dhcp, ssh, etc.). I have not tried to log "flow" as it will overkill my setup due to the complex network that is analysing.

All the other logged events are shown except the netflow events, which i am certain they are being logged

[jasonish]

I'm thinking something must be mangling your netflow events along the way. Under the events tab, all the queries including netflow are just very simple term queries on the event_typefield and don't really care about anything else. If TLS logs are displayed for instance, then netflow logs should be as well, provided they are in the same index.

The only real way to debug this right now is to use the discover tool in Kibana to make sure the data as you expect it to be. I currently log both flow and netflow.

Re flow vs netflow, flow logs half as many records, that can be useful high volume environments.

[ngms17]

I noticed that I didn't explain myself well and I apologize for that.

My problem is that I can't see any Netflow event in the reports on "Reports" -> "Netflow" section.

[jasonish]

Do you see netflow events under "Events" -> "Netflow" though?

[ngms17]

Yes, under "Events" -> "Netflow" i can see the events

[jasonish]

What version of Elasticsearch? And there are no custom mappings added to the index?

[ngms17]

Currently using 7.17.3 version. In the meantime, i disabled "netflow" logging and enabled only "flow" logging. Now i can see data in "Reports" -> "Flow" however i cannot see any data in "Traffic ID" and "Traffic Labels" tables.

I am using Filebeat Suricata module default mappings and nothing was changed

[jasonish]

Currently using 7.17.3 version. In the meantime, i disabled "netflow" logging and enabled only "flow" logging. Now i can see data in "Reports" -> "Flow" however i cannot see any data in "Traffic ID" and "Traffic Labels" tables.

These require the OISF traffic ID ruleset. If using it already the Filebeat Suricata module may be interfering, see below.

I am using Filebeat Suricata module default mappings and nothing was changed

Ok, that is probably the issue. The Filebeat Suricata module does a lot of remapping, and I probably have not caught every case yet.

[ngms17]

Currently using 7.17.3 version. In the meantime, i disabled "netflow" logging and enabled only "flow" logging. Now i can see data in "Reports" -> "Flow" however i cannot see any data in "Traffic ID" and "Traffic Labels" tables.

These require the OISF traffic ID ruleset. If using it already the Filebeat Suricata module may be interfering, see below.

Is what i suspected. Will activate that ruleset.

I am using Filebeat Suricata module default mappings and nothing was changed

Ok, that is probably the issue. The Filebeat Suricata module does a lot of remapping, and I probably have not caught every case yet.

So i need to create some custom mapping for it to work?

[jasonish]

So i need to create some custom mapping for it to work?

Not much you can do on your end. When you start EveBox with the --ecs flag it adapts it queries for ECS style events. Its obviously not handling netflow events correctly, so thats more query adaptation required inside of Suricata.

[jasonish]

I've pushed a fix to master to give a report on what was easy to add for now. I do hope to enrich this as time allows. Not sure when I'll tag the next release tho.