search

HelenGuohx / logbert

log anomaly detection via BERT
MIT License
240 stars 102 forks source link

Results are far from the values reported in the paper #9

Closed luisfredgs closed 3 years ago

luisfredgs commented 3 years ago

My results are far from the values reported in the paper. For example, after running the code on the BGL dataset I've gotten:

Precision: 88.93%, Recall: 79.86%, F1-measure: 84.15%

Running on HDFS, I've gotten:

Precision: 4.67%, Recall: 79.67%, F1-measure: 8.83%

What happening? Is there anything I missed?

ChangNamAn commented 3 years ago

My results with default parameter are: BGL 2K TP: 57, TN: 602, FP: 86, FN: 20 Precision: 39.86%, Recall: 74.03%, F1-measure: 51.82%

HDFS TP: 6600, TN: 0, FP: 0, FN: 4047 Precision: 100.00%, Recall: 61.99%, F1-measure: 76.54%

Note that those results were using the previous code before setting up this repository. (Downloaded the source code as a zip file)

HelenGuohx commented 3 years ago

Sorry about that, I am checking the code. this is not the original code. this is a modified code

ChangNamAn commented 3 years ago

I got the result with custom log data as below. Precision: 53.81%, Recall: 98.82%, F1-measure: 69.68% And tbird TP: 269374, TN: 220414, FP: 1782, FN: 39235 Precision: 99.34%, Recall: 87.29%, F1-measure: 92.93%

HelenGuohx commented 3 years ago

I uploaded an older version of code.