search

aquanauts / alogamous

A log anomaly detection framework
MIT License
1 stars 9 forks source link

Line counters being unable to recognize lines #57

Closed kitrady closed 1 month ago

kitrady commented 1 month ago

For the analyzer that count log lines of a specific level, they might be unable to count lines they need to if the "level" section of the log line is formatted in different ways between log files. For example, if the analyzer is searching for the string "error", but the lines use the string "ERROR" as the level, the analyzer would miss all the error lines. My ideas for dealing with are either by checking for several variants (eg "ERROR" and "error" and "Error"), or by just choosing one and maybe marking lines that don't use it as improperly formatted, but @laurenassour might have better ideas.

kitrady commented 1 month ago

I think I saw this problem with the info analyzer, but I could be misremembering

EDIT: I checked and I believe this line from the info analyzer has this problem since it is searching for "info" but the example log files use "INFO"

kitrady commented 1 month ago

Another instance of this issues comes up with the warning analyzer, as some log files use "warning" and some use "warn", so that is an additional consideration for that level of message