Standard input/output from the commands that are run in an atomic test are missing.

[ironmanstittsville]

I have been writing a powershell script that has a list of commands to run (each one of them being Invoke-AtomicTest with a unique Technique/TestNumber).

Throughout this script I am producing log entries to the user Temp directory. The log entries are things like:

• Began running tests at 11:48 am on July 16, 2020
• I have 25 atomic tests to run
• Test 1/25 has now started ...
• Test 1/25 has finished.

My issue is that I have no output from the commands that the test case are running (the elipsis between 'started' and 'finished' above are empty). This is not optimal for me because I wish to run these test cases regularly and then run detection on the generated telemetry. This helps me to have confidence in my detection software as I modify it. If the test case failed then I'd like to know why and fix it.

I'd like to run this powershell script once per day (for example) and then scan the log file to ensure that it all ran as I expect it to. The problem that I have is that atomic test cases can fail silently and there is no log information about what failed and when and why...

What I want is to have a log file that looks like this:

[datetime] Began running test 1/25
[datetime]       Test: T1197 Testnumber 3
BITSADMIN version 3.0
...
Created job {abcdef12-12345678-...}
...
[datetime] Finished running test 1/25
[datetime] Began running test 2/25
...

I am producing all of the lines with the [datetime] in them and expected that when I called Invoke-AtomicTest, it would return the stdout in a variable OR I could pipe the output of the command to my log file (appending to what is already there).

This wasn't working (I'm quite new to Powershell so I figured that I had done something wrong) so I dug into the code. I found that this flow occurs, when I call Invoke-AtomicTest: Invoke-AtomicTest -> Invoke-ExecuteCommand -> Invoke-Process At the bottom of that flow, the Invoke-Process, I can see that the standard out and error are lost.

I made some changes that allow me to: Invoke-AtomicTest T1197 -TestNumbers 3 -LogFileName "C:\Users\john\AppData\Local\Temp\2020-07-16-214534.txt

I've modified the call flow so that it:

• passes that filename all the way down to the Invoke-Process where
• it will capture the stdout and stderr to files (ART-stdout.txt and ART-stderr.txt)
• when the process is finished, it will Get-Content those files and append them to the log file name that has been passed down.

It works pretty well. As I was working on that this week I was thinking that I may not be the only person who would like to have complete logs after I run a test.

I will try to do a pull and submit my code for your review. I'm sure that there is a better way to do this (by somehow flowing the stdout/stderr back up to the caller) but I'm not skilled enough to pull that off.

I think that the below a useful use case to support. The output would be a log file with my progress messages as well as the stdout/stderr from the commands being run in the Atomic test.

"run_tests.ps1" $List-Of-Tests = @{} $List-Of-Tests += { Invoke-AtomicTest T1197 -TestNumbers 3 } .... Add-Content $ALOGFILENAME -value "Starting my tests" foreach ( $COMMAND in $List-Of-Tests ) { Invoke-Command -ScriptBlock $COMMAND | Out-File $ALOGFILENAME -Append }

[ironmanstittsville]

I'm on the slack channel, btw, as RichardH.

[clr2of8]

Hi Richard, thanks for all the info and work. I'll pick up a conversation with you over on Slack.

[ironmanstittsville]

So, I have a fix coded. It is a different, and better, fix than what I describe above.

I have made a change that simply captures the stdout and stderr into a file (actually, I stole this solution based on @clr2of8 advice) and then write that output to stdout for the caller. So, the caller can just pipe the output to a log file if they wish and they will have the output from the commands that were run).

This soln is less code change and it is much cleaner.

[ironmanstittsville]

Code review performed. I need to add in an 'interactive' option to the Invoke-AtomicTest command that will bypass this stdout/stderr processing for those tests that must be interactive. I'll code and test this tomorrow and submit a PR for review.

[ironmanstittsville]

Pull request submitted. I've tested it in my scenarios. Please let me know if the code needs some changes. I've got an environment setup for dev/test.

[clr2of8]

Pull request approved. Thank you!