Multiple issues with Atomic Red Team plugin

[PSR009]

Issue#1 : Though the test case (ability) has passed with given output, the status is shown as failed.

• Example 1.a : T1021.001 - AtomicTest#2 - Changing RDP Port to Non Standard Port via Powershell
• Link Command : Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp' -name "PortNumber" -Value 4489; New-NetFirewallRule -DisplayName 'RDPPORTLatest-TCP-In' -Profile 'Public' -Direction Inbound -Action Allow -Protocol TCP -LocalPort 4489
• Link Output :

  Name                          : {5ccc8a2e-11fa-4404-a5cb-123233f6c11d}
  DisplayName                   : RDPPORTLatest-TCP-In
  Description                   : 
  DisplayGroup                  : 
  Group                         : 
  Enabled                       : True
  Profile                       : Public
  Platform                      : {}
  Direction                     : Inbound
  Action                        : Allow
  EdgeTraversalPolicy           : Block
  LooseSourceMapping            : False
  LocalOnlyMapping              : False
  Owner                         : 
  PrimaryStatus                 : OK
  Status                        : The rule was parsed successfully from the store. (65536)
  EnforcementStatus             : NotApplicable
  PolicyStoreSource             : PersistentStore
  PolicyStoreSourceType         : Local
  RemoteDynamicKeywordAddresses : 

• During execution, a warning is shown as WARNING (atomic_powershell.py:12 parse) This ability failed for some reason. Manually updating the link to report a failed state. but is there a way to know this reason or debug and resolve it?
• Example 1.b : T1021.002 - AtomicTest#2 - Map Admin Share PowerShell
• Link Command : New-PSDrive -name g -psprovider filesystem -root \\Target\C$
• Link Output :

  Name           Used (GB)     Free (GB) Provider      Root                                               CurrentLocation
  ----           ---------     --------- --------      ----                                               ---------------
  g                                      FileSystem    \\Target\C$                                                       

Issue#2 : Prerequisite commands are not executed in the ability, leading to multiple failures.

• Example 2.a : T1021.002 - AtomicTest#3 - Copy and Execute File with PsExec
• Link Command : C:\PSTools\PsExec.exe \\localhost -accepteula -c C:\Windows\System32\cmd.exe
• Link Output : The system cannot find the path specified.
• Is manually adding an executor (in necessary abilities) with prerequisite commands the only solution or it's due to a setup failure?
• No icon/symbol is shown under the Requires column in the adversary profile.
• Example 2.b : T1021.006 - AtomicTest#3 - WinRM Access with Evil-WinRM
• Link Command : evil-winrm -i Target -u Domain\Administrator -p P@ssw0rd1
• Link Output :

  
  evil-winrm : The term 'evil-winrm' is not recognized as the name of a cmdlet, function, script file, or operable 
  program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
  At line:1 char:1

• evil-winrm -i Target -u Domain\Administrator -p P@ssw0rd1

• 
  + CategoryInfo          : ObjectNotFound: (evil-winrm:String) [], CommandNotFoundException
  + FullyQualifiedErrorId : CommandNotFoundException
  ```

Issue#3 : Some abilities are not executed during an operation, they are not shown as failed or error.

• Example 3.a : T1021.001 - AtomicTest#1 - RDP to DomainController

Screenshots for all 3 issues mentioned above

• https://pasteboard.co/WsU8jtcKfBf1.png
• https://pasteboard.co/xxCzIcifofdF.png

Desktop

• OS : CentOS Linux 7
• Browser : Chrome
• Version : 4.1.0
• Agent : Windows 10 Pro 21H2

[github-actions[bot]]

Looks like your first issue -- we aim to respond to issues as quickly as possible. In the meantime, check out our documentation here: http://caldera.readthedocs.io/

[github-actions[bot]]

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 5 days

[elegantmoose]

@PSR009 RE Issue 1 - The error status is coming from this general post-ability parser (https://github.com/mitre/atomic/blob/master/app/parsers/atomic_powershell.py#L6). Currently, Im not sure why the simple error check (for all atomic powershell abilities) is to check for the line 'FullyQualifiedErrorId' in the ability output, and I dont have a test windows box where I can test this ability. Lets ping the former team member who created the parser a few years ago.

@ArtificialErmine - any chance youre active and can remember why we created the parser this way?

@PSR009 - If you want to get hacky, try removing the check for 'FullyQualifiedErrorId' from that parser in the source code and see what happens.

[elegantmoose]

@PSR009 RE Issue 2 - These are Atomic Red Teams abilities (as opposed to Caldera's) so we dont maintain those.

However, if you wanted to fix, you can just create a Caldera ability that does the needed actions before this ability.

[ArtificialErmine]

Yep, @elegantmoose, I'm still around. Let's see what I can pull out here.

For Issue 1 - I believe we were using FullyQualifiedErrorId because a plurality of error cases we tested produced that kind of error code when failing. Originally, we didn't have any reliable way to detect any error, so we implemented this to detect at least some of them. Unfortunately, due to the fact that atomic tests are super wide spread in terms of what they actually do, it's hard to get a constantly valid error code across every test to check for. That being said, I would double check that the command isn't producing some text that isn't being captured somehow, and if it isn't, then I'd have someone on the team double check that the comparison is working as expected (it might be checking for the presence of the characters of the string FullyQualifiedErrorId rather than the string as a whole).

For Issue 2 - As @elegantmoose pointed out, we don't maintain the Atomic Red Team techniques per say, just load them and provide a platform to interact with. As a result, while we make a best effort to load Pre-Requisites (atomic red team style) and handle them, we can't make a guarantee that it works for every case, and it looks like this is one that slips through the current system.

For Issue 3 - My guess is that some Pre-Requisites (caldera style) for the ability aren't being met. I'd need to look into the actual abilities files to be sure of that, but that's my initial guess as to why they wouldn't be executed.

[elegantmoose]

@PSR009 RE issue 3 - yes exactly what @ArtificialErmine said

[github-actions[bot]]

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 5 days