search

activecm / ipfix-rita

Collect IPFIX / Netflow v9 Records and Ship them to RITA for Analysis
https://www.activecountermeasures.com/
10 stars 1 forks source link

Solve IPFIX issue with MikroTik Router Logs #58

Closed SamuelCarroll closed 5 years ago

SamuelCarroll commented 5 years ago

Currently it appears that there is an issue reading the IPFIX Logs incoming from the MikroTik router. The error follows ERRO[1082] input map must contain key 'netflow.flowStartMilliseconds'

The input map follows: map[ _id:ObjectIdHex(\"5c004115354bf5fc1800008c\") netflow:map[ tcpControlBits:16 destinationTransportPort:80 destinationMacAddress:REDACTED postSourceMacAddress:REDACTED tcpWindowSize:4096 udpMessageLength:0 ingressInterface:2 flowEndSysUpTime:85237060 isMulticast:0 ipTotalLength:40 sourceIPv4Address:REDACTED ipTTL:64 ipHeaderLength:5 octetDeltaCount:559 postNATSourceIPv4Address:REDACTED postNATDestinationIPv4Address:REDACTED ipNextHopIPv4Address:REDACTED version:10 destinationIPv4Address:REDACTED tcpAcknowledgementNumber:256845980 sourceIPv4PrefixLength:0 icmpCodeIPv4:0 flowStartSysUpTime:85222260 icmpTypeIPv4:0 packetDeltaCount:3 ipVersion:4 ipClassOfService:0 postNAPTSourceTransportPort:54507 sourceTransportPort:54507 destinationIPv4PrefixLength:0 egressInterface:0 protocolIdentifier:6 postNAPTDestinationTransportPort:80 igmpType:0 tcpSequenceNumber:661187750 ] @timestamp:\"2018-11-29T19:42:11.000Z\" @version:1 host:REDACTED ]

What appears to be happening is that we expect a flowStartMilliseconds field but in the above we have flowStartSysUpTime. In input/mgologstash/flow.go on line 133 we search the map for flowStartMilliseconds, and report an error if we don't have it. Also we check for flowEndMilliseconds on line 142. It would be ideal to add a check for the flowStartSysUpTime and flowEndSysUpTime to resolve this issue.

Zalgo2462 commented 5 years ago

This is a duplicate of issue #40.

SamuelCarroll commented 5 years ago

Also looks like issues #19