search

mapto / sprks

Security Policy Risk Simulator: An exploratory simulation game featuring the real-life complexities of corporate information security
MIT License
2 stars 2 forks source link

Irrelevant incidents being generated #138

Open Zhanelya opened 11 years ago

Zhanelya commented 11 years ago

Once the policyUpdate array is passed to server side at the beginning of the month, a set of incidents is being generated.

Probably resolved: One of the reasons for irrelevant events might be that for the moment only the policy which came first in the array of updated policies with its unique combination of employee-location-device is taken to the account. This policy is then passed for the generation of the incidents list, which takes this policy and applies it for all the possible combinations of employee-location-device for both of risk types and then returns the list of incidents

Not resolved: Prediction of incidents works inappropriately in model_sklearn.py line "event = self.get_prediction(datapoints, employee, location, device, next_risk)"

Zhanelya commented 11 years ago

oracle.py - Lasagna code bit, especially sim_model().request(policies) seems to be erroneous -> simulation.py - next = self.update(policy, single_context, next_risk) seems to return irrelevant list of events

Zhanelya commented 11 years ago

if no policy is set for a combination of employee-location-device, for unresolved reason the predicted event seem to be chosen to be the most frequent one

Zhanelya commented 11 years ago

@mapto What is the difference between classifier_sklearn and regression_sklearn when predict_single_datapoint is called from them? the 1st one gets event name the other the probability of this event to happen? if so, how is the first one done and why fingerprints event seems to be one of the most probable events overriding occasionally any other events?

mapto commented 11 years ago

Initially the risk probability value was meant to be the probability belonging to the incident. However naturally this resulted in a finite set of possible probabilities (i.e. some policy parameters don't seem to affect the probability), which is not how risk probabilities in real life work.

The fingerprint event appearing in other contexts is a problem of the training of the implicit model. I think this issue is about the model not returning the values that it should.

A different issue is if the policies provided as input to the model are trivial (i.e. if default is not the default policy, but the zero policy). Still this shouldn't lead to an incident related to the use of finger scanners