nProbe: Incorrect flows being generated.

[bluefangs]

nProbe version: v.8.6.190325 (r6322) for x86_64-pc-linux-gnu

nProbe command:

$ nprobe -i eth1 -n 127.0.0.1:2055 -W -G -b 2 -V 9 -t 600 -d 10 -l 10 -T "%IN_BYTES %IN_PKTS
%OUT_BYTES %OUT_PKTS %IPV4_SRC_ADDR %IPV4_DST_ADDR %IPV4_NEXT_HOP
%L4_SRC_PORT %L4_DST_PORT %SRC_VLAN %DOT1Q_SRC_VLAN %SRC_TOS
%TCP_FLAGS %PROTOCOL %IP_PROTOCOL_VERSION %DIRECTION
%FLOW_START_MILLISECONDS %FLOW_END_MILLISECONDS %INPUT_SNMP
%OUTPUT_SNMP %IN_SRC_MAC %OUT_DST_MAC %ICMP_TYPE %BIFLOW_DIRECTION
%L7_PROTO %L7_PROTO_NAME %ICMP_IPV4_TYPE %ICMP_IPV4_CODE"

Problems observed:

1 - Spanning Tree packets are treated as IPv6 packets(HOPOPT/0) in emitted flows 2 - Sometimes flows for such packets have abnormally large IN_BYTES/OUT_BYTES 3 - Flows with truncated Application names, Wrong IP versions (0,7 or 111) etc. 4 - Some flows end up with a negative time

PCAP sample stp-packet.pcapng.zip

Supporting screenshots

[lucaderi]

A fix for 802.3 dissection has been integrated an a new build will be available within one hour from now. Please test it and report back.

[bluefangs]

Hello - Can you please confirm which version of nprobe this fix is released in?

As of now we are running the below version - and notice that the issue is still persisting. Another thing to note is - if you look at our original command from the first post, we are using the -W options - which should filter out IPv6 traffic. But it seems that nprobe is not respecting it.

nprobe --version

Welcome to nProbe v.8.6.190416 (r6328) for x86_64-pc-linux-gnu
with native PF_RING acceleration.
Copyright 2002-18 ntop.org

Build OS:      Ubuntu 16.04.6 LTS
SystemID:      798E6DD79206A1D8
GIT rev:       8.6-stable:18741f1d27ce51ff8a17028493147ca33d5c9d50:20190416

Sample flow:

{
  "_index": "nprobe-2019.04.19",
  "_type": "_doc",
  "_id": "VHgKNWoBljYEyUMiL8W1",
  "_version": 1,
  "_score": null,
  "_source": {
    "IN_BYTES": 1764398320,
    "IN_PKTS": 1,
    "OUT_BYTES": 651,
    "OUT_PKTS": 2,
    "IPV4_SRC_ADDR": "0.0.0.0",
    "IPV4_DST_ADDR": "0.0.0.0",
    "IPV4_NEXT_HOP": "0.0.0.0",
    "L4_SRC_PORT": 0,
    "L4_DST_PORT": 0,
    "SRC_VLAN": 0,
    "DOT1Q_SRC_VLAN": 0,
    "SRC_TOS": 0,
    "TCP_FLAGS": 0,
    "PROTOCOL": 0,
    "IP_PROTOCOL_VERSION": 6,
    "DIRECTION": 0,
    "FLOW_START_MILLISECONDS": 1555668013421,
    "FLOW_END_MILLISECONDS": 1555668017421,
    "INPUT_SNMP": 0,
    "OUTPUT_SNMP": 0,
    "IN_SRC_MAC": "00:1F:33:FD:2C:44",
    "OUT_DST_MAC": "01:80:C2:00:00:00",
    "ICMP_TYPE": 0,
    "BIFLOW_DIRECTION": 1,
    "L7_PROTO": "0",
    "L7_PROTO_NAME": "Unknown",
    "ICMP_IPV4_TYPE": 0,
    "ICMP_IPV4_CODE": 0,
    "IPV6_SRC_ADDR": "::",
    "IPV6_DST_ADDR": "::",
    "IPV6_SRC_MASK": 0,
    "IPV6_DST_MASK": 0,
    "@version": "1",
    "@timestamp": "2019-04-19T10:00:13.421Z",
    "NPROBE_IPV4_ADDRESS": "10.10.10.100"
  },
  "fields": {
    "@timestamp": [
      "2019-04-19T10:00:13.421Z"
    ]
  },
  "sort": [
    1555668013421
  ]
}

[emanuele-f]

Please check out the development version of ntopng (version 8.7). Install the nightly repo by following the instructions at http://packages.ntop.org

[simonemainardi]

@bluefangs did you have a chance to test the suggested version with the fix?

[bluefangs]

Apologie for getting back to this after a long period.

Our current version is:

$ nprobe --version

Welcome to nProbe v.8.7.190610 (r6510) for x86_64-pc-linux-gnu
with native PF_RING acceleration.
Copyright 2002-19 ntop.org

Build OS:      Ubuntu 16.04.6 LTS
SystemID:      798E6DD79206A1D8
GIT rev:       dev:ebaf0b34b484c2627f70a54ba00402b7faea0102:20190610
License:       Invalid nProbe license (/etc/nprobe.license) [Missing license file]

nProbe is subject to the terms and conditions defined in
the LICENSE and EULA files that are part of this package.

nProbe also contains third party code:
Radix tree code - (C) The Regents of the University of Michigan
                      ("The Regents") and Merit Network, Inc.
sFlow collector - (C) InMon Inc.

We confirm that the 802.3 dissection fix is doing its job fine. On the sidenote, the -W parameter still sends out flows pertaining to IPv6. It's not an issue for us, so If you'd like, I can open a different ticket. But for now, we are good to close this one out.

Thanks.

[emanuele-f]

Can you provide a pcap file?