VECTR is a tool that facilitates tracking of your red and blue team testing activities to measure detection and prevention capabilities across different attack scenarios
1.37k
stars
162
forks
source link
Outcome options are overly interdependent on one another #249
Describe the bug
When selecting outcome labels for a given test case, it is not possible to select every unique combination of blocked, alerted, and logged. For example, if I have a test case that is blocked by a host-based firewall that does not generate an alert when it takes that action, but there is some other event related to that action, I have no way to log this outcome (ie. blocked + logged, but not alerted). Similarly, I can't select both blocked and alerted. While I understand that this may be intended, these are separate concepts in my mind.
Blocked refers to the inability to get the expected outcome/output from some test case because a specific protective control worked as intended. Alerted refers to instances where a test case may or may not have successfully executed, but there was an alert that would be presented to an analyst for triage regardless. Logged refers to the generation of non-alert telemetry related to the execution of a test case (regardless of whether that test case was blocked and/or alerted).
To Reproduce
Steps to reproduce the behavior:
Go to the view for an arbitrary test case
Click on Alerted
Attempt to also click on Blocked
The "Alerted" box is now unchecked.
Expected behavior
With the above definitions of blocked, alerted, and logged in mind, we should be able to record these three outcomes independently of one another. It would also be nice to be able to tie defensive products to blocked, alerted, and logged outcomes independently (ex. blocked by host-based FW, but alerted by MDE).
Desktop (please complete the following information):
This is absolutely an issue we'd like to tackle, we are planning this for 2024, it is a little involved with the changes required, but expect to see this sometime next year.
Describe the bug When selecting outcome labels for a given test case, it is not possible to select every unique combination of blocked, alerted, and logged. For example, if I have a test case that is blocked by a host-based firewall that does not generate an alert when it takes that action, but there is some other event related to that action, I have no way to log this outcome (ie. blocked + logged, but not alerted). Similarly, I can't select both blocked and alerted. While I understand that this may be intended, these are separate concepts in my mind.
Blocked refers to the inability to get the expected outcome/output from some test case because a specific protective control worked as intended. Alerted refers to instances where a test case may or may not have successfully executed, but there was an alert that would be presented to an analyst for triage regardless. Logged refers to the generation of non-alert telemetry related to the execution of a test case (regardless of whether that test case was blocked and/or alerted).
To Reproduce Steps to reproduce the behavior:
Expected behavior With the above definitions of blocked, alerted, and logged in mind, we should be able to record these three outcomes independently of one another. It would also be nice to be able to tie defensive products to blocked, alerted, and logged outcomes independently (ex. blocked by host-based FW, but alerted by MDE).
Desktop (please complete the following information):