Inconsistent pipeline thread stack size

[fflaten]

Prerequisites

• [X] Write a descriptive title.
• [X] Make sure you are able to repro it on the latest released version
• [X] Search the existing issues.
• [X] Refer to the FAQ.
• [X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

Example 1:

1. Run Start-Job { function recurse([int]$i) { $i; recurse ($i+1) }; recurse 0 } | Receive-Job -Wait
2. Run function recurse([int]$i) { $i; recurse ($i+1) }; recurse 0

Example 2 on Windows: Run the following block directly in Powershell session and then inside a job

$signature = @'
[DllImport("kernel32.dll")]
public static extern void GetCurrentThreadStackLimits(out uint lowLimit, out uint highLimit);
'@

$type = Add-Type -MemberDefinition $signature -Name Win32Pinvoke -PassThru

$low, $high = 0, 0
$type::GetCurrentThreadStackLimits([ref]$low, [ref]$high)
($high - $low)/1MB

Expected behavior

Example 1: Recurse-loop fails on somewhat similar depth.
Example 2: Same/similar stack size reported

Actual behavior

Example 1: Job stops on 1215. Local pipeline (step 2) stops/freezes on 9000+
Example 2: Job reports 1,5MB. Local pipeline reports 9,5MB stack size. 

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.2.3
PSEdition                      Core
GitCommitId                    7.2.3
OS                             Microsoft Windows 10.0.22000
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

[fflaten]

6224 brought the 10MB default from Windows PowerShell for local pipeline into PowerShell. Foreach-Object -Parallel also use the 10MB DefaultPipelineStackSize

Any reason why the same limit isn't used for jobs and remote sessions?

[iSazonov]

I think it makes sense to have the same limit.

/cc @PaulHigin

[PaulHigin]

@WG-Remoting

Remote execution of commands and script is more complex than with the local pipeline case. By default the remoting endpoint will run commands/script on the same thread the remoting message is processed on, to save the overhead of generating extra threads. In the Start-Job case, these are .Net threadpool threads and you cannot change the stack size on them AFAIK.

There is a mechanism to change this default behavior and specify a PSThreadOptions.UseNewThread which will cause the remoting endpoint to execute the command/script in a new PS pipeline thread, which will automatically get the 10MB stack size. However, this is not currently exposed through Start-Job, but is something that could be added.

[fflaten]

I still believe this is worth a fix to provide a consistent environment for users regardless of how they invoke their code.

[fflaten]

@iSazonov Could we please reopen this? Still relevant and I don't like duplicating issues 🙂

[fflaten]

By default the remoting endpoint will run commands/script on the same thread the remoting message is processed on, to save the overhead of generating extra threads.

@PaulHigin Thanks for the previous answer. Adding an option to Start-Job would definitely help.

Could you elaborate a little about the overhead with using an extra thread for remote sessions? Is it a <1sec delayed session start? Higher memory usage?

Or maybe easier, why the default thread for remote processing can't have increased stack size itself?