search

zentures / sequence

(Unmaintained) High performance sequential log analyzer and parser
http://sequencer.io
517 stars 72 forks source link

time not recognized at end of line #4

Closed cryptix closed 9 years ago

cryptix commented 9 years ago

Hi,

do you have an idea why these rules don't match these messages?

msg:
Jan 31 21:42:59 mail postfix/anvil[14606]: statistics: max connection rate 1/60s for (smtp:5.5.5.5) at Jan 31 21:39:37
Jan 31 21:42:59 mail postfix/anvil[14606]: statistics: max connection count 1 for (smtp:5.5.5.5) at Jan 31 21:39:37
Jan 31 21:42:59 mail postfix/anvil[14606]: statistics: max cache size 1 at Jan 31 21:39:37

rules:
%msgtime% %apphost% %appname%[%integer%]: statistics: max connection rate %string% for (smtp:%appipv4%) at %time%
%msgtime% %apphost% %appname%[%integer%]: statistics: max connection count %integer% for (smtp:%appipv4%) at %time%
%msgtime% %apphost% %appname%[%integer%]: statistics: max cache size %integer% at %time%

the rules work perfectly, except for the %time% at the end.

alexzorin commented 9 years ago

Okay so I ran into this as well and narrowed it down to https://github.com/strace/sequence/blob/cdf9721c45774d990e7222bedc7cb303140c2cd3/scanner.go#L276-L280

Because the scanner runs out of tokens, that block of code never runs if the %time% field is the last field in the log line.

You can demonstrate this by adding another token to the end of the line, and it starts working :-1:

I don't understand the scanner that well so not sure as to the best fix, sorry

zhenjl commented 9 years ago

So @alexzorin is correct. The scanner ran out of tokens to evaluate so never got to the the leaf of the timeFSM. I added a check before returning the final token to see if the last token is a time token.

Also added these messages to both scanner_test.go and parser_test.go for future testing.

Thanks for checking this out and reporting the bug!