pfSense and ELSA

[GoogleCodeExporter]

Has anyone had any success sending logs from pfSense v2.1 into ELSA?

I have tried this script in this article without success.

http://www.securitygrit.com/2013/03/pfsense-into-elsa.html

Thanks

Original issue reported on code.google.com by bbcan...@gmail.com on 22 Nov 2013 at 11:42

[GoogleCodeExporter]

Some days ago I posted my solution to the pfSense forum:

https://forum.pfsense.org/index.php/topic,69544.msg396314.html#msg396314

Now I want to share this also in this forum.

The logs from pfSense for ICMP packets (and ESP, IGMP maybe other protocols as 
well) have more than one space in front of the ip address part (after applying 
the pfSense "oneline" patch for the remote syslog). Therefore you need 
additional patterns in the patterndb.xml file of elsa, i.e.

for "class 2" - (FIREWALL_ACCESS_DENY)

<pattern>@ESTRING:: block in on @@ESTRING:s0:: (@@ESTRING::proto @@ESTRING:i0: 
@@ESTRING::   @@IPv4:i1:@@ESTRING:: @@ESTRING:: @@IPv4:i3:@@ANYSTRING@</pattern>

and for "class 3" - (FIREWALL_CONNECTION_END)

<pattern>@ESTRING:: pass in on @@ESTRING:s0:: (@@ESTRING::proto @@ESTRING:i0: 
@@ESTRING::   @@IPv4:i1:@@ESTRING:: @@ESTRING:: @@IPv4:i3:@@ANYSTRING@</pattern>

There is an additional 'problem' with the pfSense logs in elsa:
The delimiter between the ip address an the port number is a "dot". This is no 
valid delimiter for the sphinx search engine.  So the search for an ip address 
isn't working in the indexed logs.

To solve this issue I have added an additional 'sed' command for external 
logging in pfSense in
/etc/inc/filter.inc to substitute those dots by a colon:

$oneline = isset($config['syslog']['pflog_oneline']) ? " | /usr/bin/sed -l -e 
'N;s/\\n //;P;D;' | /usr/bin/sed -l -e 's/\\(.* \
\)\\(\\([0-9]\\{1,3\\}\\.\\)\\{3\\}[0-9]\\{1,3\\}\\)\\.\\([0-9]\\{1,5\\}\\)\\( 
.* \\)\\(\\([0-9]\\{1,3\\}\\.\\)\\{3\\}[0-9]\\{1,3\\}\\)
\\.\\([0-9]\\{1,5\\}\\)\\(.*\\)/\\1\\2:\\4\\5\\6:\\8\\9/' " : " ";

Maybe there is a better solution than this "sed" command (but it is working).

Original comment by thhi.1...@gmail.com on 27 Feb 2014 at 4:30

[GoogleCodeExporter]

I forgot to mention that the patterns for the modified pfSense logs

http://www.securitygrit.com/2013/03/pfsense-into-elsa.html

must then be adapted - change "dot" to "colon" twice on multiple patterns i.e.

@IPv4:i1:@.@NUMBER:i2: @ --> @IPv4:i1:@:@NUMBER:i2: @

Original comment by thhi.1...@gmail.com on 27 Feb 2014 at 7:00