search

NYU-HSRN-Network-Data-Science-Group / AutoZeekWatch

An online, deployable machine learning network intrusion detection system for Zeek.
MIT License
3 stars 0 forks source link

Implement KitNET model inference #19

Closed diego-lopez8 closed 7 months ago

diego-lopez8 commented 7 months ago

The latest commits by @zoe70416 have made the training of the model now possible. Please implement, in the infer.py script, first apply process_json() to the json stream, (the line object in the main function) and then implement a function score_json() that takes in that processed numpy array and applies the model to it. Please return a new vector that includes the original vector as well as the anomaly score.

We will also need to somehow retain the source ip, dest ip, source port, dest port, and protocol in each vector that has the anomalies, as these are necessary to actually identify the flow in question that has the associated anomaly score. This may take implementation changes across multiple functions. Please implement this as well and track progress and discussion in this ticket.

Zihang-Xia commented 7 months ago

I have implemented the infer script. It can now detect changes continuously and predict a score on the conn.log in the current directory. Since the model can predict and train at the same time, I used the fit_score_partial function. Currently, it will update the model as it processes data.

I have included the original JSON string to retain the source IP information for each anomaly. We should discuss the exact format and attributes needed.

diego-lopez8 commented 7 months ago

closed by #22