HelenGuohx / logbert

log anomaly detection via BERT
MIT License
240 stars 102 forks source link

How can I compute the anomalous score? #12

Closed ChangNamAn closed 2 years ago

ChangNamAn commented 3 years ago

I want to deploy LogBERT but there is no related code.

Below is an explanation of it on the paper.

After training, we can deploy LogBERT for anomalous log sequence detection. The idea of applying LogBERT for log anomaly detection is that since LogBERT is trained on normal log sequences, it can achieve high prediction accuracy on predicting the masked log keys if a testing log sequence is normal. Hence, we can derive the anomalous score of a log sequence based on the prediction results on the MASK tokens.

Could you guide me on how to compute the anomalous score?

HelenGuohx commented 2 years ago

One way to do that is to calculate the number of abnormal tokens / number of masked tokens for a log sequence. The prediction results on MASK token is a probability distribution over all the log keys.