Hello KAIROS team,
Kudos on a great repository and paper.
I've recently conducted an extended evaluation of KAIROS on the DARPA TC CADETS dataset and wanted to share my findings regarding the original evaluation presented in the paper.
Original Evaluation Limitations:
Only days 6-7 out of the 10-day engagement were evaluated.
Only 1 out of 3 attacks was considered.
Extended Evaluation Approach:
Evaluated days 6-12 of the dataset.
Included all 3 attacks in the ground truth.
Key Findings:
Time Window Level:
Precision decreased to 26.32%
Recall decreased to 62.5%
Accuracy remained high at 97.34%
Was there a specific reason for limiting the evaluation to days 6-7 and only one attack?
I'm happy to share more details about my evaluation process if it would be helpful.
Hello KAIROS team, Kudos on a great repository and paper.
I've recently conducted an extended evaluation of KAIROS on the DARPA TC CADETS dataset and wanted to share my findings regarding the original evaluation presented in the paper. Original Evaluation Limitations:
Extended Evaluation Approach:
Key Findings: Time Window Level: Precision decreased to 26.32% Recall decreased to 62.5% Accuracy remained high at 97.34%
Additionally, edges level results decreased significantly.
Was there a specific reason for limiting the evaluation to days 6-7 and only one attack? I'm happy to share more details about my evaluation process if it would be helpful.