search

secdev / scapy

Scapy: the Python-based interactive packet manipulation program & library.
https://scapy.net
GNU General Public License v2.0
10.56k stars 2.01k forks source link

NetflowTemplateFieldV9 cannot decode IPFIX template contains enterprise-specific Information Element #2236

Closed ir193 closed 4 years ago

ir193 commented 5 years ago

Brief description

IPFIX is now re-using NetflowTemplateFieldV9 for template records. But IPFIX extends the format of Netflow protocol. According to IPFIX RFC, the first bit of Information Element id indicates whether it is IANA-assigned or enterprise-specific Information Elements. 

     |0| Information Element id. 1.2 |        Field Length 1.2       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             ...               |              ...              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |1| Information Element id. 1.N |        Field Length 1.N       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Enterprise Number  1.N                     |

NetflowTemplateFieldV9 in scapy now can only decode IANA-assigned template.

Environment

How to reproduce

will upload a pcap later.

ir193 commented 5 years ago

Hi here is the minimal code to reproduce the problem.

bind_bottom_up(UDP, NetflowHeader, dport=4739)
s=b'\x00\x00\x03\x04\x00\x06\x00\x00\x00\x00\x00\x00\x00\x00\x08\x00E\x00\x00\x8c\x196@\x00@\x11\xe0\xad\x01\x01\x01\x01\xac\x18\x92c\xc2\xb8\x12\x83\x00xA\x07\x00\n\x00p]r\xa7\r\x00\x00\t$\x00\x00\x00\x01\x00\x02\x00`\x01\x07\x00\x12\x01\n\x00\x04\x84\x0c\x00\x02\x00\x00\x00\t\x01\n\x00&\x00\x0b\x00\x02\x00\x07\x00\x02\x00\x04\x00\x01\x00\x0c\x00\x04\x00\x08\x00\x04\x00\xea\x00\x02\x01\n\x00\x01\x84\x10\x00\x06\x00\x00\x00\t\x84\x0e\x00\x06\x00\x00\x00\t\x84\x0f\x00\x06\x00\x00\x00\t\x00\x01\x00\x04\x00\x02\x00\x04\x00\xf3\x00\x02\x00\x06\x00\x01\x01\n\x00#'
p = CookedLinux(s)

The output when decode it using scapy is:

<CookedLinux  pkttype=unicast lladdrtype=0x304 lladdrlen=6 src='' proto=IPv4 |
<IP  version=4 ihl=5 tos=0x0 len=140 id=6454 flags=DF frag=0 ttl=64 proto=udp chksum=0xe0ad src=1.1.1.1 dst=172.24.146.99 |
<UDP  sport=49848 dport=ipfix len=120 chksum=0x4107 |<NetflowHeader  version=10 |
<NetflowHeaderV10  length=112 ExportTime=Fri, 06 Sep 2019 18:35:57 -0800 (1567794957) flowSequence=2340 ObservationDomainID=1 |
<NetflowFlowsetV9  flowSetID=2 length=96 templates=[
    <NetflowTemplateV9  templateID=263 fieldCount=18 template_fields=[
        <NetflowTemplateFieldV9  fieldType=opaqueOctets fieldLength=4 |>, 
        <NetflowTemplateFieldV9  fieldType=33804 fieldLength=2 |>, 
        <NetflowTemplateFieldV9  fieldType=0 fieldLength=9 |>, 
        <NetflowTemplateFieldV9  fieldType=opaqueOctets fieldLength=38 |>, 
        <NetflowTemplateFieldV9  fieldType=L4_DST_PORT fieldLength=2 |>, 
        <NetflowTemplateFieldV9  fieldType=L4_SRC_PORT fieldLength=2 |>, 
        <NetflowTemplateFieldV9  fieldType=PROTOCOL fieldLength=1 |>, 
        <NetflowTemplateFieldV9  fieldType=IPV4_DST_ADDR fieldLength=4 |>, 
        <NetflowTemplateFieldV9  fieldType=IPV4_SRC_ADDR fieldLength=4 |>, 
        <NetflowTemplateFieldV9  fieldType=ingressVRFID fieldLength=2 |>, 
        <NetflowTemplateFieldV9  fieldType=opaqueOctets fieldLength=1 |>, 
        <NetflowTemplateFieldV9  fieldType=33808 fieldLength=6 |>, 
        <NetflowTemplateFieldV9  fieldType=0 fieldLength=9 |>, 
        <NetflowTemplateFieldV9  fieldType=33806 fieldLength=6 |>, 
        <NetflowTemplateFieldV9  fieldType=0 fieldLength=9 |>, 
        <NetflowTemplateFieldV9  fieldType=33807 fieldLength=6 |>, 
        <NetflowTemplateFieldV9  fieldType=0 fieldLength=9 |>, 
        <NetflowTemplateFieldV9  fieldType=IN_BYTES fieldLength=4 |>] |>, 
    <NetflowTemplateV9  templateID=2 fieldCount=4 template_fields=[
        <NetflowTemplateFieldV9  fieldType=dot1qVlanId fieldLength=2 |>, 
        <NetflowTemplateFieldV9  fieldType=TCP_FLAGS fieldLength=1 |>, 
        <NetflowTemplateFieldV9  fieldType=opaqueOctets fieldLength=35 |>] |>] |>>>>>>

Decoding with Wireshark, the output is 

Cisco NetFlow/IPFIX
    Version: 10
    Length: 112
    Timestamp: Sep  6, 2019 11:35:57.000000000 PDT
    FlowSequence: 2340
    Observation Domain Id: 1
    Set 1 [id=2] (Data Template): 263
        FlowSet Id: Data Template (V10 [IPFIX]) (2)
        FlowSet Length: 96
        Template (Id = 263, Count = 18)
            Template Id: 263
            Field Count: 18
            Field (1/18): opaqueOctets
            Field (2/18): 1036 [pen: ciscoSystems]
                1... .... .... .... = Pen provided: Yes
                .000 0100 0000 1100 = Type: 1036 [pen: ciscoSystems]
                Length: 2
                PEN: ciscoSystems (9)
            Field (3/18): opaqueOctets
            Field (4/18): L4_DST_PORT
            Field (5/18): L4_SRC_PORT
            Field (6/18): PROTOCOL
            Field (7/18): IP_DST_ADDR
            Field (8/18): IP_SRC_ADDR
            Field (9/18): ingressVRFID
            Field (10/18): opaqueOctets
            Field (11/18): 1040 [pen: ciscoSystems]
            Field (12/18): 1038 [pen: ciscoSystems]
            Field (13/18): 1039 [pen: ciscoSystems]
            Field (14/18): BYTES
            Field (15/18): PKTS
            Field (16/18): dot1qVlanId
            Field (17/18): TCP_FLAGS
            Field (18/18): opaqueOctets

The problem happenes in the 2nd element, when first bit of field_type is set to 1, there is an extra field for enterprise number. For exmaple, the first element of this template in hex is 01 0a 00 04, 01 0a is opaqueOctets(266), 04 is lenght. The seconde element in hex is 84 0c 00 02 00 00 00 09, the first bit is set, length is 2, enterprise number is 00 00 00 09 meaning ciscoSystem. Scapy only decodes the first 4 bytes, and then trys to decode the remaining 4 bytes as another template field.

I think it can be easily fixed here https://github.com/secdev/scapy/blob/master/scapy/layers/netflow.py#L1331 . I'm writing a patch but don't know how to define a optional field. Could someone provide me an exmaple I can follow?