search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.35k stars 1.06k forks source link

Graylog 3.1 CEF UDP regression #6922

Open guiguiabloc opened 4 years ago

guiguiabloc commented 4 years ago

Since migrate to Graylog 3.1, the CEF UDP input was unable to decode message (works fine in 2.4).

Expected Behavior

Decode CEF Message from Checkpoint Firewall

CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|sqlnet1|Unknown|act=Drop deviceDirection=0 rt=1575387805000 spt=51690 dpt=1521 cs2Label=Rule Name layer_name=Prod_fw Security layer_uuid=927f269b-52dd-4cab-8550-7aeffda6ca02 match_id=4019 parent_rule=0 rule_action=Drop rule_uid=3e517596-1229-c34a-86e4-ab3a7f3ff752 ifname=bond1.45 logid=0 loguid={0x0,0x0,0x0,0x0} origin=10.254.4.203 originsicname=CN=fw,O=fw.net.m9ug26 sequencenum=292 version=5 dst=10.140.27.121 inzone=Internal outzone=Internal product=VPN-1 & FireWall-1 proto=6 service_id=sqlnet1 src=10.241.45.1

Current Behavior

Error in decode process

2019-12-03T16:52:09.226+01:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=dcf8ef90-15e4-11ea-b6a8-0050568a1266, journalOffset=-9223372036854775808, codec=CEF, payloadSize=19, timestamp=2019-12-03T15:52:09.225Z, remoteAddress=/10.145.24.247:38344} java.lang.NullPointerException: null at org.graylog.plugins.cef.parser.MappedMessage.<init>(MappedMessage.java:37) ~[graylog.jar:?] at org.graylog.plugins.cef.codec.CEFCodec.decodeCEF(CEFCodec.java:128) ~[graylog.jar:?] at org.graylog.plugins.cef.codec.CEFCodec.decode(CEFCodec.java:117) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:86) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:45) [graylog.jar:?] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]

Steps to Reproduce (for bugs)

1.Add CEF input 2.send CEF message from Checkpoint Firewall log manager 3.Error in Graylog log (in 2.4 no error)

Context

Your Environment

jalogisch commented 4 years ago

He @guiguiabloc

could you please provide some log messages from the specific device and the version numbers. It is not very likely that we have that device and if the same software version. But that are information we need to determine if the problem is only with this device or a general.

guiguiabloc commented 4 years ago

Hi @jan

Checkpoint Firewall-1 Version r80.20

Below some messages (i change some IPs):

`message CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|domain-udp_|Unknown|act=Accept destinationTranslatedAddress=0.0.0.0 destinationTranslatedPort=0 deviceDirection=0 rt=1575387866000 sourceTranslatedAddress=1.1.1.1 sourceTranslatedPort=0 spt=38349 dpt=53 cs2Label=Rule Name layer_name=Prod_fw Security layer_uuid=b5116248-bf7d-48a7-ae84-a5e18c65fc88 match_id=72 parent_rule=0 rule_action=Accept rule_uid=ae817241-6c19-fb4b-a46d-8904927594f7 ifname=bond1.3 logid=0 loguid={0x5de682da,0x10031,0x2801fa0a,0xc0000005} origin=10.250.1.40 originsicname=CN\=fw-dmz,O\=fw.m9ug26 sequencenum=85 version=5 dst=25.21.193.164 inzone=Internal nat_addtnl_rulenum=1 nat_rulenum=55 outzone=External product=VPN-1 & FireWall-1 proto=17 serviceid=domain-udp src=192.168.72.25

message CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|https_|Unknown|act=Accept destinationTranslatedAddress=0.0.0.0 destinationTranslatedPort=0 deviceDirection=0 rt=1575387866000 sourceTranslatedAddress=1.1.1.1 sourceTranslatedPort=0 spt=33973 dpt=443 cs2Label=Rule Name layer_name=Prod_fw Security layer_uuid=b5116248-bf7d-48a7-ae84-a5e18c65fc88 match_id=141 parent_rule=0 rule_action=Accept rule_uid=b1b6334c-2009-3a4d-98cf-f140a982b68c ifname=bond1.3 logid=0 loguid={0x5de682da,0x10034,0x2801fa0a,0xc0000005} origin=10.250.1.40 originsicname=CN\=fw-dmz,O\=fw.m9ug26 sequencenum=93 version=5 dst=22.2.2.2 inzone=Internal nat_addtnl_rulenum=1 nat_rulenum=54 outzone=External product=VPN-1 & FireWall-1 proto=6 serviceid=https src=192.168.72.7

message CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|domain-udp_|Unknown|act=Accept destinationTranslatedAddress=0.0.0.0 destinationTranslatedPort=0 deviceDirection=0 rt=1575387866000 sourceTranslatedAddress=1.1.1.1 sourceTranslatedPort=0 spt=40842 dpt=53 cs2Label=Rule Name layer_name=Prod_fw Security layer_uuid=b5116248-bf7d-48a7-ae84-a5e18c65fc88 match_id=72 parent_rule=0 rule_action=Accept rule_uid=ae817241-6c19-fb4b-a46d-8904927594f7 ifname=bond1.3 logid=0 loguid={0x5de682da,0x1006b,0x2801fa0a,0xc0000003} origin=10.250.1.40 originsicname=CN\=fw-dmz,O\=fw.m9ug26 sequencenum=66 version=5 dst=2.2.2.2 inzone=Internal nat_addtnl_rulenum=1 nat_rulenum=55 outzone=External product=VPN-1 & FireWall-1 proto=17 serviceid=domain-udp src=192.168.72.25

message CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|TCP-7778|Unknown|act=Drop deviceDirection=0 rt=1575387866000 spt=41994 dpt=7778 cs2Label=Rule Name layer_name=Prod_fw Security layer_uuid=b5116248-bf7d-48a7-ae84-a5e18c65fc88 match_id=327 parent_rule=0 rule_action=Drop rule_uid=f52ec25b-5654-4d49-bd78-23710328660a ifname=eth0 logid=0 loguid={0x0,0x0,0x0,0x0} origin=10.250.1.40 originsicname=CN\=fw-dmz,O\=fw.m9ug26 sequencenum=100 version=5 dst=1.1.1.1 inzone=External outzone=External product=VPN-1 & FireWall-1 proto=6 service_id=TCP-7778 src=2.2.2.2 `

jan commented 4 years ago

@guiguiabloc thanks for your contribution, bra. You're doing a great job.