SecurityRiskAdvisors / VECTR

VECTR is a tool that facilitates tracking of your red and blue team testing activities to measure detection and prevention capabilities across different attack scenarios
1.37k stars 162 forks source link

Outcome options are overly interdependent on one another #249

Open luv2crawl opened 11 months ago

luv2crawl commented 11 months ago

Describe the bug When selecting outcome labels for a given test case, it is not possible to select every unique combination of blocked, alerted, and logged. For example, if I have a test case that is blocked by a host-based firewall that does not generate an alert when it takes that action, but there is some other event related to that action, I have no way to log this outcome (ie. blocked + logged, but not alerted). Similarly, I can't select both blocked and alerted. While I understand that this may be intended, these are separate concepts in my mind.

Blocked refers to the inability to get the expected outcome/output from some test case because a specific protective control worked as intended. Alerted refers to instances where a test case may or may not have successfully executed, but there was an alert that would be presented to an analyst for triage regardless. Logged refers to the generation of non-alert telemetry related to the execution of a test case (regardless of whether that test case was blocked and/or alerted).

To Reproduce Steps to reproduce the behavior:

  1. Go to the view for an arbitrary test case
  2. Click on Alerted
  3. Attempt to also click on Blocked
  4. The "Alerted" box is now unchecked.

Expected behavior With the above definitions of blocked, alerted, and logged in mind, we should be able to record these three outcomes independently of one another. It would also be nice to be able to tie defensive products to blocked, alerted, and logged outcomes independently (ex. blocked by host-based FW, but alerted by MDE).

Desktop (please complete the following information):

p-b-- commented 11 months ago

This is absolutely an issue we'd like to tackle, we are planning this for 2024, it is a little involved with the changes required, but expect to see this sometime next year.

luv2crawl commented 11 months ago

Thanks @p-b-- for the quick response!