Experiment Plot 2 (metric vs threshold)

[divyapathak24]

• Threshold: No. of consecutive instances flagged as abnormal
• FPs = Normal class being classified as attack
• FNs = Attack class being classified as normal

Objective : What is the (right threshold, contamination parameter combination) to keep FPR small and FNR close to zero? Why this plot? To get the right threshold value for which FPR is least for some ballpark contamination parameter values on which IF is being trained on

Plot 2.a - FPR vs threshold vs contamination parameters Plot 2.b - FNR vs threshold vs contamination parameters

Details:

• Metric: FPR/FNR
• X-axis: Threshold (1,2,3,4,5) - [No of consecutive instances]
• Y-axis: FPR/FNR (FPR/FNR for chosen contamination parameter - [0.001,0.005,0.01,0.05,0.1])

[divyapathak24]

• This plot has been completed for bmv2 where the IF is trained on feature stats represented using Function 1
• This plot is pending for CAIDA experiment

[praveenabt]

@divyapathak24 post the plots we have for bmv2 and document the key observations.

[divyapathak24]

BMV2 Plots: Note:

• Here, contamination parameter and rates are in fractions i.e., 0.1 denotes 10%
• Plots are for 20 experiments (collection of instances) comprised of: a) 6 normal experiments (3 normal, 3 packet loss 5%, 10%, 12%) b) 7 normal link failure experiments - influences FPR
  c) 7 attack case experiments - influences FNR
• It is better to pick a threshold of lesser value at the cost of some penalty in terms of FPR/FNR

-FPR vs threshold vs contamination parameters Key observations:

• If one can tolerate some FPRs for contamination parameters values 0.05 and 0.1, we can pick threshold 2

-FNR vs threshold vs contamination parameters Key observations:

• If one can tolerate some FNRs for contamination parameters values 0.001 and 0.005, we can pick threshold 3