search

wuyifan18 / DeepLog

Pytorch Implementation of DeepLog.
MIT License
361 stars 154 forks source link

fix validation #33

Closed d0ng1ee closed 4 years ago

d0ng1ee commented 4 years ago

Use hashmap for acceleration and ensure the true validation score, score:0.89->0.9435, still lower than the origin paper 0.96

wuyifan18 commented 4 years ago

@donglee-afar hi, do you have a try to use the 'list' not 'set' of the dataset when inferencing?

d0ng1ee commented 4 years ago

@wuyifan18 After 3h30min testing, the result which use the 'list' not 'set' of the dataset is the same as 0.9435, and there is no bug in my new predict code...

wuyifan18 commented 4 years ago

@donglee-afar I just run it and get the result below. You can try a few more times. image

d0ng1ee commented 4 years ago

@wuyifan18 You are right, after used MultiStepLR strategy and train with some more epoch, the result of model is stable with 0.95~0.952, and the val data in your repo seems are a few differences from the original data... Example, there is only 28 templates in your repo...Maybe you use Drain extract log templates instead of ground truth templates(29 templates)??? But this is a minor problem and i will try to implement other log anomaly detection algorithms. Thank you for your repo to help me know the field quickly!

wuyifan18 commented 4 years ago

@donglee-afar the data is given by the author in HERE. I just use it and it is not extracted by me.