search

SecurityRiskAdvisors / VECTR

VECTR is a tool that facilitates tracking of your red and blue team testing activities to measure detection and prevention capabilities across different attack scenarios
1.35k stars 159 forks source link

Imported Atomic test cases automation do not include prereq commands #218

Open chryzsh opened 1 year ago

chryzsh commented 1 year ago

Describe the bug After importing and trying to use Atomic Red Team (ART) cases into Vectr I noticed. This occasionally breaks where a prereq command is required for the test case to run. Prereq command in atomic are often used to set env vars, prepare a path or similar before the automation itself runs. Cases that run without the prereq command will often fail, leading to a large amount of imported test cases not working as intended. They are similar to the (already supported in Vectr) cleanup commands.

Example: The atomic test for "Encrypt files using 7z (Linux)" has a prereq command for setting an env var that references the binary the test cases uses in execution. Here is a direct link to the yaml with the prereq command: https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1486/T1486.yaml#L69

The test case automation today looks like this, and fails when running because the env var which_7z is never set. image

Here is a screenshot of how the log looks. It has received an error because the env var is not set, and its thus trying to execute the case without it, making it think the first argument is the command to run. image

If I either set the env var on the command line before running the automation, or add a line before the test case command with that command, it works as expected.

To Reproduce Steps to reproduce the behavior:

  1. Import Atomic test cases intro Vectr
  2. Go to an existing campaign or create a new one
  3. Add a test case from ART that contains automation and a cleanup command. See my example above for a specific one.
  4. Click on a test case and "Configure" to see the automation configuration for it. Note how it is missing the prereq command.
  5. Generate the build automation shell script and execute it.
  6. See error

Expected behavior I would expect the prereq commands to be added to the automation and be executed before the test itself. Also, note that cleanup commands from ART are supported and working in the automation runtime in Vectr already.

Desktop (please complete the following information): OS: Windows 10 Browser: Chrome Version: ce-8.7.1

Additional context A possible workaround is to look up and manually add prereq commands in the automation, or on the command line before executing the tests. This is however a laborous manual task which does not scale and reduces the value of using the automation runtime.

thebleucheese commented 1 year ago

This is a good idea. It's a bigger effort than some of the other automation/RTA requests because it requires data model changes. The Atomic Red Team project added this after we'd completed initial dev. We'll likely need to hold off on implementing these prereqs until after we've made some larger VECTR data model changes expected over the next 3-4 months.