How to convert the parsed data to training data?

[Gharibim]

Thank you so much for your efforts! Would you please help with this issue, I think it is somehow repeated but I wanted to clear the confusion. After we get the logs, I used SPELL parser and I end up with dictionary looks like this:

{ "0": {"log_id": [10,2,3], "abstraction": "......"},
  "1":{"log_id": [0,2,7,9,10], "abstraction": "......"}
...
....
}

I used another SPELL parser and got this result:

0[]
1[]
0[['203519'], ['/10.250.10.6:40524'], ['/10.250.10.6:50010']]
0[['203519', '145'], ['/10.250.14.224:42420'], ['/10.250.14.224:50010']]
...
...

What is the next step ? I mean how to come up with the training data now? test_normal and test_abnormal?

[wenqibiao]

@Gharibim Have you any new progress? I have the same problem.

[wuyifan18]

You can use Drain which is proposed in this paper to extract log template and convert these templates to ID.

[haosanzi]

@wuyifan18 Can you tell me how to convert log templates to training data? Thank you very much!

[wuyifan18]

@haosanzi I just convert training data to numbers using templates ID from 0 to length of templates.

[ying1016]

@Gharibim Could you tell me where did you find the first SPELL? The link is not available now. Or could you tell me how you fixed the output of the second SPELL? My results looked the same as the second. I hope to get the result as your first one and do the next step. Thank you!

[errorhandlerst]

@ying1016 you can get the result (0, '{"lineids": [1, 2, 3], "postion": [2], "lcsseq": "this is pen "}') (1, '{"lineids": [4, 5, 6, 7], "postion": [2], "lcsseq": "i am "}') via second spell which is suggested by @Gharibim via executing command cat train.log | python slm_create.py