bitkeks / python-netflow-v9-softflowd

PyPI "netflow" package. NetFlow v9 parser, collector and analyzer implemented in Python 3. Developed and tested with softflowd
https://bitkeks.eu/blog/2016/08/collecting-netflow-v9-on-openwrt.html
MIT License
110 stars 56 forks source link

KeyError: 'IP_PROTOCOL_VERSION' #3

Closed sm3ulc closed 6 years ago

sm3ulc commented 6 years ago

Get some kind of parsing error:

python3 analyze_json.py 1519108140.json Traceback (most recent call last): File "analyze_json.py", line 133, in con = Connection(pending, flow) File "analyze_json.py", line 48, in init ips = getIPs(src) File "analyze_json.py", line 22, in getIPs if flow['IP_PROTOCOL_VERSION'] == 4: KeyError: 'IP_PROTOCOL_VERSION'

Example from json-file:

{"IPV4_SRC_PREFIX": 0, "LAST_SWITCHED": 2397666550, "L4_SRC_PORT": 47597, "PROTOCOL": 6, "IN_BYTES": 436, "TCP_FLAGS": 25, "SRC_MASK": 0, "INPUT_SNMP": 52, "IPV4_DST_ADDR": 2249824527, "IPV4_SRC_ADDR": 2249835426, "FLOW_SAMPLER_ID": 7, "FIRST_SWITCHED": 2397666400, "DST_AS": 0, "DIRECTION": 1, "OUTPUT_SNMP": 52, "IPV4_NEXT_HOP": 2887713825, "SRC_AS": 0, "IN_PKTS": 1, "DST_MASK": 0, "L4_DST_PORT": 80, "SRC_TOS": 0}

Debug from main.py looks "fine":

Received data from x.y.z.n, length 372 Processed ExportPacket with 6 flows. ...

Output is from a Cisco 2T-sup and is working in pmacct etc.

// David

bitkeks commented 6 years ago

Thanks for your report @sm3ulc! As you can see in your JSON dict, there is no "IP_PROTOCOL_VERSION": x field inside, so the parser in analyze_json cannot find it to check the version. A solution would be to check for the field first and if this fails, use fields like IPV4_SRC_ADDR because they indicate IPv4 usage.

The reason for this bug might be that your exporter chooses different fields to export than softflowd, which I used for exporting my flows. Meaning, if you use a Cisco-internal exporter it may choose to not use the field IP_PROTOCOL_VERSION in its exports.