What steps will reproduce the problem?
1.) mysql syslog: create class (10053,"bind_dns"), fields.dns_{date,time,client,query,resolver}, fields_classes_map.{167..171} tied to new class id (10053)
classes:
id , class
10053 , BIND_DNS
fields:
id , field , field_type , pattern_type
167 , dns_date , string , ESTRING
168 , dns_time , string , ESTRING
169 , dns_client , int , IPv4
170 , dns_query , string , ESTRING
171 , dns_resolver , int , IPv4
fields_classes_map:
field_id , field_order
167 , 11
168 , 12
169 , 5
170 , 13
171 , 6
2.) syslog-ng.conf: add source udp(), destination /var/log/dns_syslog.log, source file("/var/log/dns_syslog.log"), put source file into log{} whose destination is d_elsa. Full syslog-ng.conf file is attached.
source s_dns_syslog{ udp(); };
source sf_dns_syslog{ file("/var/log/dns_syslog.log"); };
log { source(sf_dns_syslog); [[bro log sources/rewrites/merged parser from Security Onion]]; destination(d_elsa); };
log { source(s_dns_syslog); destination(df_dns_syslog); };
3.) Create parser, merge with Security Onion's patterndb.xml (pdbtool checks out clean)
root@server:/etc/elsa/patterns.d/dns_syslog# pdbtool -v test dns_syslog_parser
Testing message program='dns_syslog' message='20-Jan-2015 09:15:45.133 queries: info: client 111.222.333.444#58425 (1.a1aa1aaaa1aa111a.geoipd.global.sonicwall.com): query: 1.a1aa1aaaa1aa111a.geoipd.global.sonicwall.com IN ANY +ED (444.333.222.111)'
Match name='.classifier.rule_id', value='10053', expected='10053'
Match name='s0', value='20-Jan-2015', expected='20-Jan-2015'
Match name='s1', value='09:15:45.133', expected='09:15:45.133'
Match name='i0', value='111.222.333.444', expected='111.222.333.444'
Match name='s2', value='1.a1aa1aaaa1aa111a.geoipd.global.sonicwall.com IN ANY +ED', expected='1.a1aa1aaaa1aa111a.geoipd.global.sonicwall.com IN ANY +ED'
Match name='i1', value='444.333.222.111', expected='444.333.222.111'
<ruleset>
<pattern>dns_syslog</pattern>
<rules>
<rule class='10053' id='10053'>
<patterns>
<pattern>@ESTRING:s0: @@ESTRING:s1: @queries: info: client @IPv4:i0:@@ESTRING::query@: @ESTRING:s2: (@@IPv4:i1:@)</pattern>
</patterns>
<examples>
<example>
<test_message program="dns_syslog">20-Jan-2015 09:15:45.133 queries: info: client 111.222.333.444#58425 (1.a1aa1aaaa1aa111a.geoipd.global.sonicwall.com): query: 1.a1aa1aaaa1aa111a.geoipd.global.sonicwall.com IN ANY +ED (444.333.222.111)</test_message>
<!-- date -->
<test_value name="s0">20-Jan-2015</test_value>
<!-- time -->
<test_value name="s1">09:15:45.133</test_value>
<!-- client -->
<test_value name="i0">111.222.333.444</test_value>
<!-- query -->
<test_value name="s2">1.a1aa1aaaa1aa111a.geoipd.global.sonicwall.com IN ANY +ED</test_value>
<!-- resolver -->
<test_value name="i1">444.333.222.111</test_value>
</example>
</examples>
</rule>
</rules>
</ruleset>
________________
________________
What is the expected output? What do you see instead?
Expected: Query for class=BIND_DNS returns DNS query logs parsed by date, time, client, query, and resolver
Actual: No results for class=BIND_DNS or program=dns_syslog, all of the logs fall under class=NONE, program=named, host=IP.AD.DR.ESS
________________
________________
What version of the product are you using? On what operating system?
ELSA:
Sphinx Sphinx 2.0.7-id64-dev (rel20-r373)
Date 2013-12-04 12:00:00 -0400 (Wed, 04 Dec 2013)
Author mcholste
Rev 1090
Security Onion 12.04.5:
Linux seconid 3.13.0-44-generic #73~precise1-Ubuntu SMP Wed Dec 17 00:39:15 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Syslog-NG:
root@server:/# syslog-ng -V
syslog-ng 3.3.4
Installer-Version: 3.3.4
Revision: /3.3.4.dfsg-2ubuntu1
Compile-Date: Apr 5 2012 17:29:34
Default-Modules: affile,afprog,afsocket,afuser,afsql,basicfuncs,csvparser,dbparser,syslogformat
Available-Modules:
dbparser,afprog,afsocket,afmongodb,basicfuncs,confgen,afsql,csvparser,afsocket-t
ls,syslogformat,convertfuncs,tfjson,afuser,affile
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: on
Enable-TCP-Wrapper: on
Enable-Linux-Caps: on
Enable-Pcre: on
________________
________________
Please provide any additional information below.
In syslog-ng.conf, I've tried both setting the direct udp(); source into the
construct with destination(d_elsa) as well as using the source file
(/var/log/dns_syslog.log)
Noticed that when I create the udp(); to dns_syslog.log, it has to come after
the d_elsa construct, or the log file doesn't populate.
The BIND_DNS class and its fields show up in ELSA's "Add Term" and "Report On"
menus, but never yield results when searched (i.e., class="BIND_DNS").
I'm fairly certain that it's not a parser syntax issue and that syslog-ng is
giving ELSA the data since it's showing up in queries. Am I missing something
in the parser that tells it on which data to act?
Original issue reported on code.google.com by sam.r.k...@gmail.com on 21 Jan 2015 at 6:28
Original issue reported on code.google.com by
sam.r.k...@gmail.comon 21 Jan 2015 at 6:28Attachments: