search

activecm / ipfix-rita

Collect IPFIX / Netflow v9 Records and Ship them to RITA for Analysis
https://www.activecountermeasures.com/
10 stars 1 forks source link

Support PFSense/ Softflowd #40

Closed Zalgo2462 closed 5 years ago

Zalgo2462 commented 5 years ago

PFSense is a common OS used for routing based on BSD. In order to support IPFIX, PFSense employs the softflowd module.

Softflowd is an open source software based netflow exporter for Linux and BSD.

Currently Unsupported IPFIX Fields

ElementID Name Abstract Data Type Data Type Semantics Status Description
21 flowEndSysUpTime unsigned32   current The relative timestamp of the last packet of this Flow. It indicates the number of milliseconds since the last (re-)initialization of the IPFIX Device (sysUpTime). sysUpTime can be calculated from systemInitTimeMilliseconds. milliseconds     [RFC5102] 1 2014-01-11
22 flowStartSysUpTime unsigned32   current The relative timestamp of the first packet of this Flow. It indicates the number of milliseconds since the last (re-)initialization of the IPFIX Device (sysUpTime). sysUpTime can be calculated from systemInitTimeMilliseconds.
160 systemInitTimeMilliseconds dateTimeMilliseconds default current The absolute timestamp of the last (re-)initialization of the IPFIX Device.

systemInitTimeMilliseconds is sent in a data flow (matching an option template) along with the IPFIX templates every 16 packets. See https://github.com/irino/softflowd/blob/master/ipfix.c#L421

Raw wireshark example dump:

Set 3 [id=3] (Options Template): 256
    FlowSet Id: Options Template (V10 [IPFIX]) (3)
    FlowSet Length: 30
    Options Template (Id = 256) (Scope Count = 1; Data Count = 4)
        Template Id: 256
        Total Field Count: 5
        Scope Field Count: 1
        Field (1/1) [Scope]: meteringProcessId
            0... .... .... .... = Pen provided: No
            .000 0000 1000 1111 = Type: meteringProcessId (143)
            Length: 4
        Field (1/4): systemInitTimeMilliseconds
            0... .... .... .... = Pen provided: No
            .000 0000 1010 0000 = Type: systemInitTimeMilliseconds (160)
            Length: 8
        Field (2/4): selectorAlgorithm
            0... .... .... .... = Pen provided: No
            .000 0001 0011 0000 = Type: selectorAlgorithm (304)
            Length: 2
        Field (3/4): samplingPacketInterval
            0... .... .... .... = Pen provided: No
            .000 0001 0011 0001 = Type: samplingPacketInterval (305)
            Length: 2
        Field (4/4): samplingPacketSpace
            0... .... .... .... = Pen provided: No
            .000 0001 0011 0010 = Type: samplingPacketSpace (306)
            Length: 4

Set 4 [id=256] (1 flows)
    FlowSet Id: (Data) (256)
    FlowSet Length: 24
    [Template Frame: 5]
    Flow 1
        Metering Process Id: 92091
        System Init Time: Oct 18, 2018 17:57:57.250000000 MDT
        Selector Algorithm: Systematic count-based Sampling (1)
        Sampling Packet Interval: 1
        Sampling Packet Space: 0
Zalgo2462 commented 5 years ago

We'll need to maintain a table from exporters to system init times. This table will need to be referenced while decoding the IPFIX BSON data.