search

wuyifan18 / DeepLog

Pytorch Implementation of DeepLog.
MIT License
374 stars 155 forks source link

How to get log key? #6

Closed hzxGoForward closed 4 years ago

hzxGoForward commented 5 years ago

I have read this parper recently, before DeepLog, I found there need to parser logs to be log keys. so I want to know is there any tools to parse logs or how do you parse logs to be log keys? if you know any orther tools or source code, please tell me, thanks a lot!

wuyifan18 commented 5 years ago

Maybe you could ask @Athiq about the Spell which is the parser mentioned in the paper.

hzxGoForward commented 5 years ago

ok，thx! There is another question that the training data ware grouped by "blk_*", do you know the reason why he grouped raw log and then train the DeepLog model on the grouped data? this step is not mentioned in his paper, In my eyes, in terms of the anomaly detection in execution path, the correct step of training step is:

  1. parse the raw log files into log keys.
  2. numbering each log key an unique number from 1 to n and encode every log key to be a one-hot vector.
  3. replace every raw log entity to it's correspending one-hot vector.
  4. setting model parameters and for each input, computing their real probability
  5. training model.

The paper has never mentioned to grouped blog by "blk_*", if you know, please tell me the reason.

amineebenamor commented 5 years ago

This website gives additional information concerning the paper: https://www.cs.utah.edu/~mind/papers/deeplog_misc.html It explains that the log entries are grouped based on their block id "blk_*", and train/test for each block.

ImenGBY commented 5 years ago

This website gives additional information concerning the paper: https://www.cs.utah.edu/~mind/papers/deeplog_misc.html It explains that the log entries are grouped based on their block id "blk_*", and train/test for each block.

hello, the the links within the blog are not accessible: http://iiis.tsinghua.edu.cn/~weixu/demobuild.zip http://iiis.tsinghua.edu.cn/~weixu/200nodes.rar, are there further links to check? thanks

lilihongjava commented 5 years ago

This website gives additional information concerning the paper: https://www.cs.utah.edu/~mind/papers/deeplog_misc.html It explains that the log entries are grouped based on their block id "blk_*", and train/test for each block.

hello, the the links within the blog are not accessible: http://iiis.tsinghua.edu.cn/~weixu/demobuild.zip http://iiis.tsinghua.edu.cn/~weixu/200nodes.rar, are there further links to check? thanks

url prefix is changed to http://people.iiis.tsinghua.edu.cn/~weixu/