search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.4k stars 1.07k forks source link

FortiGate Netflow data not being interpreted correctly #4925

Closed xdroop closed 5 years ago

xdroop commented 6 years ago

Expected Behavior

Netflow parsed by built-in netflow plugin.

Current Behavior

The IP addresses parsed from the fields are incorrect.

Example parsed record:

Timestamp | nf_dst | nf_src | nf_start | nf_stop | nf_xlate_dst_addr_ipv4 | nf_xlate_src_addr_ipv4
-- | -- | -- | -- | -- | -- | --
2018-07-19 14:53:15.000 | 0.12.76.64:62140 | 68.0.0.0:6789 | 2018-07-19T18:53:12.720Z | 2018-07-19T18:53:13.750Z | 129.172.30.1 | 3.173.32.183
NetFlowV9 [68.0.0.0]:6789 <> [0.12.76.64]:62140 proto:6 pkts:5 bytes:1540

When I examine the nf_xlate_XXX_addr_ipv4 fields, it looks like the IP addresses are byte-shifted somehow -- my internal subnet which is where the netflow traffic being examined is generated is 172.30.1.0/24. Note this appears partially in nf_xlate_dst_addr_ipv4 above.

Steps to Reproduce (for bugs)

Netflow input on Graylog: port 2056, all other settings default

Fortigate configuration:

fgt300d-a (global) # config sys netfl

fgt300d-a (netflow) # show
config system netflow
    set collector-ip 172.30.1.57
    set collector-port 2056
    set source-ip 192.168.255.253
    set active-flow-timeout 1
end

config system interface
    edit "MyInterface"
        [...]
        set netflow-sampler both
        [...]

...note that this interface is a VLAN interface, in case that matters.

Context

Graylog is correctly parsing netflows generated by fprobe on a linux system, so I have two netflow inputs set up at the moment -- one for the fprobe generated flows and one for the direct-from-the-fortigate flows.

I'm trying to get a more searchable interface for my Fortigate netflows than is provided by nfsen. Also if I can generate netflows right off of the firewall I don't need an external, wire-sniffing system to generate netflows. So primarily I care about source IP/port, destination IP/port, and bytes.

This may not entirely be a Graylog issue -- we were never able to get nfcap/nfdump/nfsen to correctly ingest Fortigate-generate netflows, which is why I still have the external system using fprobe.

I have/will attach a pcap file showing some captured netflows from the fortigate.

fgt300d-netflow.zip

Your Environment

ebnozn commented 5 years ago

I can also confirm this issue with Graylog 2.5 and sending flows from a Fortigate 900D. I send our flows to both Graylog and NFSen simultaneously, and am attaching a text file here to show how the NFSen (correctly) reads the source/dest IPs, and how Graylog parsed the exact same flow. It's causing us to not be able to use the threat intel plugins etc. due to false positives.

Graylog 2.5 Fortigate Netflow parsing issue.txt

daniel-velinov commented 5 years ago

Same thing happens to me on using Graylog 2.5.1 and Fortigates 140d and 600D configured to send Netflow statistics to graylog. All the IPs processed by Graylog have the following IPs. Can someone please provide update on this fix

NetFlowV9 [68.0.0.0]:5060 <> [0.12.4.64]:56434 proto:17 pkts:12 bytes:8796

daniel-velinov commented 5 years ago

In my log file I have many of those errors

2018-12-29T21:55:58.492+02:00 ERROR [NetFlowCodec] Error parsing NetFlow packet received from <10.10.195.1:1657> org.graylog.plugins.netflow.flows.InvalidFlowVersionException: Invalid NetFlow version 2304 at org.graylog.plugins.netflow.v5.NetFlowV5Parser.parseHeader(NetFlowV5Parser.java:67) ~[graylog-plugin-netflow-2.5.1.jar:?] at org.graylog.plugins.netflow.v5.NetFlowV5Parser.parsePacket(NetFlowV5Parser.java:33) ~[graylog-plugin-netflow-2.5.1.jar:?] at org.graylog.plugins.netflow.codecs.NetFlowCodec.decodeMessages(NetFlowCodec.java:127) [graylog-plugin-netflow-2.5.1.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:148) [graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]

mpfz0r commented 5 years ago

Invalid NetFlow version 2304

Ugh, this seems to be a byte ordering problem. 2304 is 0x0900 in network byte order.

dio99 commented 5 years ago

same in graylog 3.0 i tried to use a yaml file that i found from logstash still dont get it ok in fields

mpfz0r commented 5 years ago

Hmm, unfortunate. I fixed all the issues I found with the .pcap that @xdroop provided. @dio99 Could you send me the errors in your graylog-server.log? Or even better a pcap of your netflow data? Make sure that it includes template records.

dio99 commented 5 years ago

@mpfz0r i can create a pcap from a fortigate if that what u need ? i got FG at home and can create the netflow to my Graylog env at home. I havent seen the errors as below but still the tmeplate wont set the propper src ip's in graylog netlow input

mpfz0r commented 5 years ago

@dio99 yes, a pcap would be useful to have. thank you.

dio99 commented 5 years ago

hmm strange i did setup netflow to send to graylog i see count but no messages is in graylog with the template from logstash/ES https://github.com/logstash-plugins/logstash-codec-netflow/blob/master/lib/logstash/codecs/netflow/netflow.yaml im using fortios version 5.6.7

graylog 3.0 ES 6.6 server.log 2019-02-25T20:15:13.495+01:00 WARN [UdpTransport] receiveBufferSize (SO_RCVBUF) for input NetFlowUdpInput{title=netflow, type=org.graylog.plugins.netflow.inputs.NetFlowUdpInput, nodeId=null} (channel [id: 0xeb1fe55f, L:/0:0:0:0:0:0:0:0%0:2099]) should be 262144 but is 425984.

tcpdump

20:22:20.759750 IP (tos 0x0, ttl 64, id 59220, offset 0, flags [DF], proto UDP (17), length 88) 192.168.1.1.2223 > 192.168.1.52.2099: [udp sum ok] UDP, length 60 20:22:26.849697 IP (tos 0x0, ttl 64, id 17239, offset 0, flags [DF], proto UDP (17), length 1336) 192.168.1.1.2223 > 192.168.1.52.2099: [udp sum ok] UDP, length 1308 20:22:29.716661 IP (tos 0x0, ttl 64, id 25432, offset 0, flags [DF], proto UDP (17), length 1328) 192.168.1.1.2223 > 192.168.1.52.2099: [udp sum ok] UDP, length 1300

dia sniffer in FG

filters=[udp and port 2099] 2.952514 192.168.1.1.2223 -> 192.168.1.52.2099: udp 248 0x0000 4500 0114 439b 4000 4011 72b8 c0a8 0101 E...C.@.@.r..... 0x0010 c0a8 0134 08af 0833 0100 2cc3 0009 0003 ...4...3..,..... 0x0020 0312 109e 5c75 9d02 0000 02a7 0000 0001 ....\u.......... 0x0030 0106 0050 0000 0000 0000 0467 0000 0000 ...P.......g.... 0x0040 0000 0467 0000 0006 0000 0006 0311 f992 ...g............ 0x0050 0311 fa6e 2383 d197 0008 0001 0614 0000 ...n#........... 0x0060 3044 0000 0006 0c8c 4003 9b04 1966 5e89 0D......@....f^. 0x0070 771d c0a8 0129 0000 0000 2383 0000 0000 w....)....#..... 0x0080 0106 0050 0000 0000 0000 054f 0000 0000 ...P.......O.... 0x0090 0000 054f 0000 0008 0000 0008 0311 f992 ...O............ 0x00a0 0311 fa6e d197 2383 0001 0008 0614 0000 ...n..#......... 0x00b0 3044 0000 0006 0c8c 4003 5e89 771d 9b04 0D......@.^.w... 0x00c0 1966 0000 0000 c0a8 0129 0000 2383 0000 .f.......)..#... 0x00d0 0102 0044 0000 0000 0000 0038 0000 0000 ...D.......8.... 0x00e0 0000 0038 0000 0001 0000 0001 030f 4706 ...8..........G. 0x00f0 030f 4846 22b8 1631 0001 0000 1114 0000 ..HF"..1........ 0x0100 3044 0000 0000 0c04 4000 d05b 714b 9b04 0D......@..[qK.. 0x0110 1966 0000 .f..

mpfz0r commented 5 years ago

@dio99 don't use a custom template. the one we ship works fine. I don't know how much this has diverged from ours (graylog2-server/src/main/resources/netflow9.yml) but this might not work.

That ASCII dump of your packet is not very useful to work with. Could you use $ tcpdump -S1500 -w fgdump.pcap and give me access to that file?

dio99 commented 5 years ago

whats -S ?

dio99 commented 5 years ago

aha seq 1500

dio99 commented 5 years ago

pcap.pcap.zip

mpfz0r commented 5 years ago

Thanks. This pcap is working fine in my unit test:

message: NetFlowV9 [40.76.209.56]:51413 <> [192.168.1.41]:9091 proto:17 pkts:1 bytes:311
message: NetFlowV9 [192.168.1.41]:9091 <> [40.76.209.56]:51413 proto:17 pkts:1 bytes:132
message: NetFlowV9 [93.78.207.99]:6881 <> [192.168.1.41]:9091 proto:17 pkts:1 bytes:337
...

There's got to be another problem in your setup..

dio99 commented 5 years ago

hmm strange.. yes i dont even get the messages in Graylog but i see that the input is reciveing it.. and offcourse the tcpdump get it no error in server.log im gonna test on another install to..