search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.4k stars 1.07k forks source link

Stream simulator says a message matches all rules but stream never matches #2494

Closed lennartkoopmann closed 8 years ago

lennartkoopmann commented 8 years ago

When trying to match this String value in the field message:

[1:1421:11] SNMP AgentX/tcp request [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.1.4:47364 -> 192.168.1.24:705

... with the regular expression ^\[\d+:\d+:\d+].*, the stream rule simulator shows the rule as matching but messages are never actually written to the stream.

lennartkoopmann commented 8 years ago

I found the issue. The message field starts with a whitespace. Here is a dump directly from ES:

"message":" [1:1418:11] SNMP request tcp [Classification: Attempted Information Leak] [Priority: 2] {TCP} 192.168.1.4:47364 -> 192.168.1.24:161\n"

The stream simulator seems to perform trimming here, which is not happening in the actual stream matcher code. IMO the most important job of the stream simulator is to resemble the actual stream matcher logic as closely as possible, or even 1:1.

A way to solve this could be to perform a trim() on any message field? I don't see a use case in which you would like to have whitespaces or tabs around field content. The way HTML handles these whitespaces (swallows them under certrain circumstances) just adds more to the confusion.

joschi commented 8 years ago

@lennartkoopmann Congratulations, you ran into #1936.

See https://github.com/Graylog2/graylog2-server/issues/1936#issuecomment-200866314 :trollface:

Closing this issue as a duplicate.