search

logpai / Drain3

A robust streaming log template miner based on the Drain algorithm
Other
444 stars 130 forks source link

Questions #11

Closed b-barry closed 4 years ago

b-barry commented 4 years ago

Hello,

Thanks for this good repo and blog post regarding log parsing. I would like to use this library to do some works and I have some remaining questions before jumping in. What do you with the template that you found? Do you transforn them to regex? I don't get the usefullness of masking. What is the difference between masking and preprocessing (as you said in your blog)? Do you plan to support other masking format such as grok ? Could share some information about the analytics pipeline? Do you forseen the upcoming blog post regarding that?

Thanks for your time and response.

Kind regards

davidohana commented 4 years ago

We convert the templates we found to multiple time-series and then perform anomaly detection on top of those. Masking improves Drain accuracy and also allows you to extract parameters and perform anomaly detection on those too. We would gladly accept a PR for grok support .. A blog post or paper about the next steps of the pipeline is planned for the next few months.

Thanks!