d0ng1ee / logdeep

log anomaly detection toolkit including DeepLog
MIT License
387 stars 115 forks source link

Use the deeplog model on streaming log? #18

Open inspurwusixuan opened 3 years ago

inspurwusixuan commented 3 years ago

Hi, thanks for this awesome toolkit!

I took a look at the BGL dataset and found that the anomaly log with the same label shares the same error message. For example, the anomaly log with type KERNDTLB shares the error message RAS KERNEL FATAL data TLB error interrupt.

KERNDTLB 1118552678 2005.06.11 R30-M0-N9-C:J16-U01 2005-06-11-22.04.38.300588 R30-M0-N9-C:J16-U01 RAS KERNEL FATAL data TLB error interrupt

So it seems to me if there is an alert trigger built on the real-time streaming log data, then multiple regular expression based rules would be enough to detect the anomaly error. So I'm wondering is there any advantage to use deeplog model on the streaming log to detect the anomaly compared to the regular expression based rules?

Any thoughts are welcomed!