Not all NetflowV9 records are defined

[static33rus]

Hi, i have problems with decoding netflow DataFlowSet: some fields are "unknown_data" and not decoded.

There is example:

[ Netflow Header ]

       version   = 9

[ Netflow Header V9 ]

          count     = 1
          sysUptime = 97568
          unixSecs  = Mon, 25 Mar 2019 08:41:58 +0000 (1553503318)
          packageSequence= 3
          SourceID  = 0

[ Netflow DataFlowSet V9 ]

             templateID= 1027
             length    = 26
             \records   \
              |###[ Netflow DataFlowset Record V9 ]### 
              |  unknown_data= b'\x10'
              |  IPV4_SRC_ADDR= 192.168.10.200
              |  unknown_data= b'\x10'
              |  PROTOCOL  = tcp
              |  unknown_data= b'\x10'
              |  unknown_data= b'\x10'
              |  unknown_data= b'\x10'

versions: scapy (2.4.3rc1.dev78) python: Python 3.6.7

Steps: 1) pkts=rdpcap("netflow3.2.pcap") 2) a=netflowv9_defragment(pkts) 3) a[1].show()

How can i get values of this fields? pcap in attachment Thanks for answer

netflow.zip

[gpotter2]

Hi. Thanks for the report. Please checkout https://github.com/secdev/scapy/pull/1965. (Please read the description)

[static33rus]

Thank you for help! Now it looks fine!

|###[ Netflow DataFlowset Record V9 ]### 
                  |  observationTimeMilliseconds= b'\x00\x00\x01i\xb4\x03p$'
                  |  IPV4_SRC_ADDR= 192.168.10.200
                  |  postNATSourceIPv4Address= b'\x05\x05\x05\x01'
                  |  PROTOCOL  = tcp
                  |  portRangeStart= b'\x04\x00'
                  |  portRangeEnd= b'\x04?'
                  |  natEvent  = b'\x10'

Will you support ipfix in future? I tried to write code for ipfix by analogy and got something like this:

###[ Netflow Header ]### 
           version   = 10
###[ Netflow Header V10 ]### 
              Length    = 90
              unixSecs  = Thu, 28 Mar 2019 15:45:39 +0000 (1553787939)
              packageSequence= 2
              ObservationDomainId= 0
###[ Netflow DataFlowSet V10 ]### 
                 templateID= 1024
                 length    = 74
                 \records   \
                  |###[ Netflow DataFlowset Record V10 ]### 
                  |  fieldValue= '\x00\x00\x01i\xc4\xfah\xf6\xc0\xa8\n\xc8\x05\x05\x05\x01\x11\x005\x01\xdb\xc0\xa8\x0b\xc8\xc0\xa8\x0b\xc8\x9c\xbd\x9c\xbd\x01\x04\x00\x00\x01i\xc4\xfamj\xc0\xa8\n\xc8\x05\x05\x05\x01\x06\xaf\xac\xb7\x94\xc0\xa8\x0b\xc9\xc0\xa8\x0b\xc9\x00P\x00P\x01\x04'

if I understood correctly i need analog of netflowv9_defragment method for ipfix, but , unfortunately, i I could't do it by myself

[gpotter2]

(Let’s keep it open until the PR is merged) :-)

[gpotter2]

Hi @static33rus, as I was fixing this PR, I ended up adding IPFix support to https://github.com/secdev/scapy/pull/1965 Could you please check it out and report how well it works ? I still need to add the tests :)

[static33rus]

Hi, Thank you for help. Today i tried to read ipfix pcap with your fix, but have got:

###[ Netflow Header ]### 
           version= 10
###[ Raw ]### 
              load= '\x00Z\\\x9c\xec#\x00\x00\x00\x02\x00\x00\x00\x00\x04\x00\x00J\x00\x00\x01i\xc4\xfah\xf6\xc0\xa8\n\xc8\x05\x05\x05\x01\x11\x005\x01\xdb\xc0\xa8\x0b\xc8\xc0\xa8\x0b\xc8\x9c\xbd\x9c\xbd\x01\x04\x00\x0

steps, which i did: 1) git clone https://github.com/gpotter2/scapy 2) execute run_scapy 3)

>>> from scapy.layers import netflow
>>> pkts=rdpcap("/home/nfware/scapy_dev/ipfix.pcap")
>>> a=netflowv9_defragment(pkts)
>>> a[1].show()

pcap in attachment

ipfix.zip

[gpotter2]

You need to checkout the branch: netflowbig

[static33rus]

after checkout on netflowbig both v9 and v10 pcap are decoded successfully:

###[ Netflow Header ]### 
           version= 10
###[ IPFix (Netflow V10) Header ]### 
              length= 90
              ExportTime= Thu, 28 Mar 2019 15:45:39 +0000 (1553787939)
              flowSequence= 2
              ObservationDomainID= 0
###[ Netflow DataFlowSet V9 ]### 
                 templateID= 1024
                 length= 74
                 \records\
                  |###[ Netflow DataFlowset Record V9 ]### 
                  |  observationTimeMilliseconds= Thu, 28 Mar 2019 15:45:39 +0000 (1553787939)
                  |  IPV4_SRC_ADDR= 192.168.10.200
                  |  postNATSourceIPv4Address= 5.5.5.1
                  |  PROTOCOL= udp
                  |  L4_SRC_PORT= 53
                  |  postNAPTSourceTransportPort= b'\x01\xdb'
                  |  IPV4_DST_ADDR= 192.168.11.200
                  |  postNATDestinationIPv4Address= 192.168.11.200
                  |  L4_DST_PORT= 40125
                  |  postNAPTDestinationTransportPort= b'\x9c\xbd'
                  |  natOriginatingAddressRealm= b'\x01'
                  |  natEvent= b'\x04'
                  |###[ Netflow DataFlowset Record V9 ]### 
                  |  observationTimeMilliseconds= Thu, 28 Mar 2019 15:45:40 +0000 (1553787940)
                  |  IPV4_SRC_ADDR= 192.168.10.200
                  |  postNATSourceIPv4Address= 5.5.5.1
                  |  PROTOCOL= tcp
                  |  L4_SRC_PORT= 44972
                  |  postNAPTSourceTransportPort= b'\xb7\x94'
                  |  IPV4_DST_ADDR= 192.168.11.201
                  |  postNATDestinationIPv4Address= 192.168.11.201
                  |  L4_DST_PORT= 80
                  |  postNAPTDestinationTransportPort= b'\x00P'
                  |  natOriginatingAddressRealm= b'\x01'
                  |  natEvent= b'\x04'

the only thing, maybe need to change "v9" to "v10" or "v9/v10" in 'Netflow DataFlowSet V9', 'Netflow DataFlowset Record V9'?