logpai / loglizer

A machine learning toolkit for log-based anomaly detection [ISSRE'16]
MIT License
1.27k stars 423 forks source link

Diploma thesis - Tensorflow - Audit logs #57

Closed sekerjak closed 4 years ago

sekerjak commented 5 years ago

Hello,

right now I am starting with writing my diploma and I am for anomaly detection, especially Tensorflow framework and probably I'll try Autoencoders. In this page I found your framework https://github.com/logpai/loglizer and in Model's table I can see, that Autoencoders are in coming state (and this is the model, which I'll be trying to implement). Could you share with me some details regarding to AutoEncoders model? I mean e.g. if you are on the half way to do it, if it's possible I am offering my help. Could you tell me more about timeline of AutoEncoder model and if it is possible to do it this way (via paper)?

Even though you have a lot of datasets and a lot of implemented models, these models just work on labeled data right? correct me if I am wrong, is it possible to use these models on unlabeled data? I think it is, but then we don't know how to evaluate them, right?

Do you have labeled Linux logs data? I saw a table https://github.com/logpai/loghub, but unfortunately it looks like you don't, but I am just curious, because I need this type of data.

Last question: If I'll try to use your models on unlabeled Linux data, it won't work and I need labeled Linux data, right?

Thank you for your response in advance.

Jakub

zhujiem commented 4 years ago

Sorry that Autoencoder is not ready yet. We welcome any pr if possible.

The linux logs are not labeled. If you run loglizer on linux, the accuracy metrics cannot be computed. But you can still get some invariants.