airpwn / enterprise-log-search-and-archive

Automatically exported from code.google.com/p/enterprise-log-search-and-archive
0 stars 0 forks source link

Please help add few new fields. #191

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1.new fields request.
2.
3.

What is the expected output? What do you see instead?
don't want add private field. be overwrite or lost upgrade ability.

What version of the product are you using? On what operating system?
Sphinx  Sphinx 2.1.3-id64-dev (r4319)
Date     2014-01-09 22:55:52 -0800 (Thu, 09 Jan 2014)
Rev  1152
OS: CentOS 6.4 x86_64

Please provide any additional information below.
Could you help add below fields into 'syslog.field' table?
+-----+---------------------+------------+--------------+------------------+
| id  | field               | field_type | pattern_type | input_validation |
        nas_ip              | int        | IPv4         | IPv4
        nac_host            | string     | QSTRING      | NULL
        eap_type_name       | string     | QSTRING      | NULL
        cmd                 | string     | QSTRING      | NULL

I want have below log been parsers, have not finished yet, just as reference.
<ruleset name='CISCO_ACS' id='6e336205485e876858cf9ccb2185b48b'>
        <pattern>CisACS</pattern>

<pattern>@ESTRING::Message-Type@@QSTRING:s0:=,@User-Name@QSTRING:s1:=,@Group-Nam
e@QSTRING:s4:=,@NAS-IP-Address@QSTRING:s2:=,@Caller-ID@QSTRING:s3:=,@EAPP Type 
Name@QSTRING::=,@</pattern>
<pattern>@ESTRING::Message-Type@@QSTRING:s0:=,@User-Name@QSTRING:s1:=,@NAS-IP-Ad
dress@QSTRING:s2:=,@Authen-Failure-Code@QSTRING:s5:=,@Caller-ID@QSTRING:s3:=,@Gr
oup-Name@QSTRING:s4:=,@</pattern>
<pattern>@ESTRING::User-Name@QSTRING:s1:=,@NAS-IP-Address@QSTRING:s2:=,@NAS-Port
@QSTRING:i0:=,@Group-Name@QSTRING:s4:=,@Framed-IP-Address@QSTRING::=,@Calling-St
ation-Id@QSTRING:s3:=,@Acct-Status-Type@QSTRING:s0:=,@Acct-Session-Id@QSTRING::=
,@</pattern>
<pattern>@ESTRING::Message-Type@@QSTRING:s0:=,@NAS-IP-Address@QSTRING:s2:=,@</pa
ttern>

96situmt 1 0 Message-Type=Authen 
failed,User-Name=xxx@xxx.xxx,NAS-IP-Address=10.1.38.120,Authen-Failure-Code=EAP-
TLS or PEAP authentication failed during SSL 
handshake,Caller-ID=10-0b-a9-aa-b9-e4,NAS-Port=13,Group-Name=Default Group,
w9htg267 1 0 
User-Name=xxx@xxx.xxx,NAS-IP-Address=10.1.38.120,NAS-Port=13,Group-Name=Group 
5,Calling-Station-Id=10.2.5.162,Acct-Status-Type=Start,Acct-Session-Id=52d84774/
00:27:10:7b:13:f8/171831,
w9htg267 1 0 
User-Name=xxx@xxx.xx,NAS-IP-Address=10.1.38.120,NAS-Port=13,Group-Name=Group 
5,Framed-IP-Address=10.2.5.162,Calling-Station-Id=10.2.5.162,Acct-Status-Type=St
art,Acct-Session-Id=52d84774/00:27:10:7b:13:f8/171831,
f458424z 1 0 User-Name=ddd,NAS-IP-Address=10.1.38.120,Group-Name=WAN 
Engineers,NAS-Portname=tty2,cmd=show running-config interface GigabitEthernet 
3/38 <cr>,priv-lvl=15,task_id=10446,

Original issue reported on code.google.com by radiu...@gmail.com on 17 Jan 2014 at 2:26

GoogleCodeExporter commented 9 years ago
sorry, the type of the case should be "Enhancement" or "Other". 

Original comment by radiu...@gmail.com on 17 Jan 2014 at 5:02

GoogleCodeExporter commented 9 years ago
I'm thinking if it's OK to add a column into syslog.fields_classes_map table. 
I'm just call it "field_display_name", actually it is 
"in_class_field_display_name". this way we can reuse existing field for 
different class. also more accurate to each type of logs.

for example there have existing field "server". 

now have a network access server(NAS), sure we can add a new fields call NAS, 
but if we can reuse existing field "server" will keep the table short, and easy 
to maintain. (not sure if this will effect performance)

+-------------+----------------------+------+-----+---------+-------+
| Field       | Type                 | Null | Key | Default | Extra |
+-------------+----------------------+------+-----+---------+-------+
| field_id    | smallint(5) unsigned | NO   | PRI | NULL    |       |
| class_id    | smallint(5) unsigned | NO   | PRI | NULL    |       |
| field_order | tinyint(3) unsigned  | NO   |     | 0       |       |
| field_display_name| varchar(255)         | NO   |     | 0       |       |

Original comment by radiu...@gmail.com on 17 Jan 2014 at 5:22

GoogleCodeExporter commented 9 years ago
You can add your own fields and they will not get overwritten during an 
upgrade. I recommend storing them in a separate, local SQL file which can be 
run like mysql -uroot syslog -e "source myfile.sql" You only need to put any 
fields which don't already exist in that file, as well as classes and the 
mapping. It could look like:

INSERT IGNORE INTO fields (field, field_type, pattern_type) VALUES ("myfield", 
"string", "QSTRING");
INSERT IGNORE INTO classes (class) VALUES ("myclass");
INSERT IGNORE INTO fields_classes_map (field_id, class_id, field_order) ...

The field_display_name column you propose is the same as changing which field 
is being referred to by the field_id, and any edits to fields_classes_map carry 
the same risk of conflict with the upstream as editing the fields table.

Original comment by mchol...@gmail.com on 18 Jan 2014 at 10:02