Enable programmatic check of process execution result

[Jil]

Usecase

Invoke-AtomicTest is used to run a sequence of tests. This sequence is chained to a sequence of API calls to the SIEM supposed to detect these activities. Then, a test report is generated.

In this context, it is important to know where the test failed: because it couldn't run on the target or because the SIEM was blind.

One needs to confirm the successful execution of Atomic Tests.

Current Implementation

Unless called with -Interactive, the standard out and error streams are directed to the temporary folder art-out.txt and art-err.txt. The content of these files are sent to the main output once the process terminated, and are deleted. The Invoke-AtomicTest cmdlet doesn't return the process exit code.

As multiple tests may be run in a single invocation, it isn't easy to provide detailed results for every one of them without adding much complexity to the code.

Suggested Implementation

Add a switch -KeepStdOutStdErrFiles to bypass the Remove-Item instruction. It is then to the caller to ensure they clean the files at the right moment (which implies to restrict invocations to a single test, otherwise the output of every test will contain the previous ones as well). This way, the caller may check standard out and error streams for trace of errors.

Tools such as mimikatz don't output their errors to stderr, that's why we may need to check both files.

The check could look like this:

if (((Test-Path -Path "${Env:TEMP}/art-err.txt") -and (Select-String "(ERROR|Exception)" "${Env:TEMP}/art-err.txt")) -or 
    ((Test-Path -Path "${Env:TEMP}/art-out.txt") -and (Select-String "(ERROR|Exception)" "${Env:TEMP}/art-out.txt"))) {
          Write-Debug ">>> Error in stream, marking test as failed in Atomic"
          [...]
}

PR on its way.

[Jil]

After playing a bit with it, the output files are cleared every time a new process is created, which lightens the burden of deleting them every time (especially in remote sessions). Additionally, calling -Clean on a test removes the files.

[Jil]

It appears some tools like procdump output to stderr what should be sent to stdout. I updated the test code sample above accordingly.

[cnotin]

That's a very good idea! I needed the same and I had to trick it 😉 (I aliased Remove-Item to nothing so it had no action)