search

patrickjennings / logstash-pfsense

Logstash configuration for pfSense syslog events.
93 stars 41 forks source link

_grokparsefailure on some lines #3

Closed a1466d44-d3dc-4c0b-90c7-315b088731d7 closed 6 years ago

a1466d44-d3dc-4c0b-90c7-315b088731d7 commented 6 years ago

Hi can you help me understand why such lines do not match the grok filter?

7,,,1000000105,igb1,match,block,in,6,0x00,0x00000,1,Options,0,32,fe80::f29f:c2ff:fe1a:2665,ff02::1,HBH,PADN,RTALERT,0x0000,
or
5,,,1000000103,ovpnc1,match,block,in,4,0x0,,64,65242,1480,none,17,udp,173,10.20.30.40,10.20.30.101,
patrickjennings commented 6 years ago

The log lines that you provided are missing the fields in the PFSENSE_PROTOCOL_DATA grok pattern.

You could make these optional. In 11-pfsense.conf:

match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}?", "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}?" ]

But you will have empty src and dest port and data length fields.

What version of pfsense are you running?

a1466d44-d3dc-4c0b-90c7-315b088731d7 commented 6 years ago

Thank you for your fast reply I'm on 2.4.1-RELEASE (amd64) built on Sun Oct 22 17:29:23 CDT 2017 FreeBSD 11.1-RELEASE-p2

PS: how are you debugging the grok pattern? The online GROK debugger where not very helpfull :)

patrickjennings commented 6 years ago

This service works best for me: http://grokconstructor.appspot.com/do/match.

In pfsense under Status / System Logs / Firewall, are you seeing any source or destination ports attached to the log entrees?

a1466d44-d3dc-4c0b-90c7-315b088731d7 commented 6 years ago

I dont have these exact entries anymore, but yes all the log entries have a source and destination. But i think this issue can be closed, as your updated GROK Pattern works perfekt ;) thx again