robcowart / elastiflow

Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Other
2.48k stars 592 forks source link

All flow records for netflow and sflow have flow.direction == 'ingress' and ipfix == 'unspecified' #174

Closed landonstewart closed 5 years ago

landonstewart commented 5 years ago

Hello,

All of our netflow and sflow records have a flow.direction of ingress (even when they are egress). Interestingly enough the ipfix records have a flow.direction of 'unspecified'.

How can we ensure the direction of flows is properly tagged? Is this a deficiency/omission on the configuration of the routers sending the netflow/sflow/ipfix records?

Also - I'm not sure if this is applicable or not but every single sflow record has the tag field set to "__sflow_ip_version_not_recognized".

landonstewart commented 5 years ago

Closing this - we'll check our routers first. If you have any comments though please feel free to add them.

modaoxia commented 5 years ago

For sflow in 20_filter_40_sflow.logstash.conf, the [flow][direction] is always set to be "ingress".

mutate {
        id => "sflow_simple_mappings"
        add_field => { "[flow][direction]" => "ingress" }
      }

I use the followings to set direction instead. You may make sure your sflow flow sample could be decoded out sflow.input_interface_value and sflow.output_interface_value by sflow codec first.

mutate {
        id => "sflow_simple_mappings"
        add_field => { "[flow][direction]" => "unspecified" }
      }

      if [sflow][source_id_index] == [sflow][input_interface_value] {
        mutate {
          id => "sflow_set_ingress_direction"
          replace => { "[flow][direction]" => "ingress" }
        }
      } else if [sflow][source_id_index] == [sflow][output_interface_value] {
        mutate {
          id => "sflow_set_egress_direction"
          replace => { "[flow][direction]" => "egress" }
        }
      }