Pipelines cause `Invoke-Expression` to work improperly

[steve02081504]

Prerequisites

• [X] Write a descriptive title.
• [X] Make sure you are able to repro it on the latest released version
• [X] Search the existing issues.
• [X] Refer to the FAQ.
• [X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

Invoke-Expression "cmd" -OutVariable ans
exit
$ans
Invoke-Expression "cmd" -OutVariable ans | Out-Host
exit
$ans

Expected behavior

PowerShell 7.4.0
PS C:\Users\steve02081504> Invoke-Expression "cmd" -OutVariable ans
Microsoft Windows [Version 10.0.19045.3803]
(c) Microsoft Corporation. All rights reserved.

C:\Users\steve02081504>exit
PS C:\Users\steve02081504> $ans
Microsoft Windows [Version 10.0.19045.3803]
(c) Microsoft Corporation. All rights reserved.

C:\Users\steve02081504>
PS C:\Users\steve02081504> Invoke-Expression "cmd" -OutVariable ans | Out-Host
Microsoft Windows [Version 10.0.19045.3803]
(c) Microsoft Corporation. All rights reserved.

C:\Users\steve02081504>exit
PS C:\Users\steve02081504> $ans
Microsoft Windows [Version 10.0.19045.3803]
(c) Microsoft Corporation. All rights reserved.

C:\Users\steve02081504>
PS C:\Users\steve02081504>

Actual behavior

PowerShell 7.4.0
PS C:\Users\steve02081504> Invoke-Expression "cmd" -OutVariable ans
Microsoft Windows [Version 10.0.19045.3803]
(c) Microsoft Corporation. All rights reserved.

C:\Users\steve02081504>exit
PS C:\Users\steve02081504> $ans
PS C:\Users\steve02081504> Invoke-Expression "cmd" -OutVariable ans | Out-Host
Microsoft Windows [Version 10.0.19045.3803]
(c) Microsoft Corporation. All rights reserved.

exit
C:\Users\steve02081504>
PS C:\Users\steve02081504> $ans
Microsoft Windows [Version 10.0.19045.3803]
(c) Microsoft Corporation. All rights reserved.

C:\Users\steve02081504>
PS C:\Users\steve02081504>

In the first Invoke-Expression the $ans variable is not updated correctly. In the second Invoke-Expression there is a problem with the cmd display: the prompt is displayed after the command is entered.

Error details

No response

Environment data

PS C:\Users\steve02081504> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.0
PSEdition                      Core
GitCommitId                    7.4.0
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

[rhubarb-geek-nz]

Try this from a simple command prompt, ( eg nothing to do with PowerShell at all )

D:\somedir>%COMSPEC% >foo.1 2>foo.2
exit

Where you have to type exit

Then

D:\somedir>type foo.1
Microsoft Windows [Version 10.0.22631.2861]
(c) Microsoft Corporation. All rights reserved.

D:\somedir>
D:\somedir>type foo.2

So you can see what was written to stdout and what was written to stderr.

exit was written to neither as it was interactive input

So I suggest it is not pipelines, but interactive input that gives you a result different to what you were expecting

[mklement0]

The behavior is arguably a bug, and it isn't specific to Invoke-Expression - Invoke-Command is equally affected.

The stand-alone Invoke-Expression "cmd" -OutVariable ans call exhibits the bug, whereas the invocation with | Out-Host command behaves correctly. However, your use of cmd without /c or /k doesn't make sense, because it enters an interactive session, which then takes over the current PowerShell session - however, since stdout is captured by Out-Host, cmd.exe's prompt string doesn't print until after a command is interactively submitted, which is why in your case it prints last.

The true bug here is that Invoke-Expression and Invoke-Command do not honor the implied request to capture stdout output when Out-Variable is used (by contrast, use of |, as in your Out-Host example, always captures, which is why the bug didn't surface there):

• The root cause is that output from native (external) programs by default bypasses PowerShell's success output stream unless it is captured - this is done both for efficiency and to allow native programs to directly control the terminal, for applications such as Vim.

• In other words: in order for Invoke-Expression and Invoke-Command to honor their -OutVariable argument consistently, they'd have to explicitly enable capturing of external-program output in stand-alone invocations.

Workarounds:

• Force capturing by appending a pipeline segment - which your ... | Out-Host call in effect did; use ... | Write-Output if you want to preserve the output as success-output.

• If up-front collection of all output lines is acceptable, enclose external-program calls in (...), which too forces capturing:

  OK, due to (...) - ditto for Invoke-Expression

  Invoke-Command -OutVariable v { (cmd /c ver) }

• Finally, you can use Tee-Object -Variable as an alternative.

  Note: There is generally no need to use Invoke-Expression to invoke external programs.

  Invoke-Expression 'cmd /c ver' | Tee-Object -Variable v

[SteveL-MSFT]

@mklement0 is correct that this is currently "by design" when native commands output isn't redirected, it's allowed to write directly to the console and PowerShell has no knowledge of it. cc @SeeminglyScience as a new experimental feature he's working on can help address this