search

proxb / PoshRSJob

Provides an alternative to PSjobs with greater performance and less overhead to run commands in the background, freeing up the console and allowing throttling on the jobs.
MIT License
541 stars 87 forks source link

Nested Jobs #169

Closed magno101 closed 6 years ago

magno101 commented 6 years ago

Do you want to request a feature or report a bug? Possibly a bug? Likely an issue with my knowledge of how to use it though.

What is the current behavior? Nesting jobs while still trying to use the $_ doesn't work as anticipated.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Please consider this code:

` $Array = @(1..(Get-Random -Maximum 10 -Minimum 2)) $Query = "Some Query String" $Start = (get-date).AddHours(-(get-date).Hour).AddMinutes(-(get-date).Minute).AddSeconds(-(get-date).Second) $End = (get-date)

Write-Host "Query: $Query" Write-Host "Start: $Start" Write-Host "End : $End"

Test the first layer

$Jobs = $Array | Start-RSJob -Name {"Layer1$"} -ScriptBlock { [pscustomobject]@{ Array = $_ Query = $Using:Query Start = $Using:Start End = $Using:End Debug = "Im in the first layer" } $NewArray = @(10..(Get-Random -Maximum 20 -Minimum 11))

$InnerJobs = $NewArray | Start-RSJob -ArgumentList $,$Using:Query,$Using:Start,$Using:End -Name {"Layer2$_"} -ScriptBlock { param( $JobNumber, $Query, $Start, $End ) [pscustomobject]@{ Array = $JobNumber Query = "2nd Layer " + $Query Start = $Start End = $End Debug = "Something at the second layer" } } $InnerJobs | Wait-RSJob | Receive-RSJob

}

$Jobs | Wait-RSJob | Receive-RSJob | ft -AutoSize `

What is the expected behavior? I would expect that the inner set of RSJobs would iterate through the $NewArray list, but instead it uses the higher scoped $_ from the original $Array.

Which versions of Powershell and which OS are affected by this issue? Did this work in previous versions of our scripts? ` Major Minor Build Revision

5 1 16299 19 1.7.3.9 PoshRSJob PSGallery OS is Windows 10 Major Minor Build Revision

10 0 16299 0 `

Please provide a code example showing the issue, if applicable: Here's the output

Query: Some Query String
Start: 10/19/2017 00:00:00
End  : 10/19/2017 11:06:44

Array Query                       Start                  End                    Debug                        
----- -----                       -----                  ---                    -----                        
    1 Some Query String           10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Im in the first layer        
    1 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    1 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    1 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    1 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    1 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    3 Some Query String           10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Im in the first layer        
    3 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    3 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    3 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    3 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    3 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    3 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    3 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    5 Some Query String           10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Im in the first layer        
    5 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    5 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    5 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    5 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 Some Query String           10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Im in the first layer        
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    4 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    6 Some Query String           10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Im in the first layer        
    6 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    6 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    6 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    6 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    6 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    6 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 Some Query String           10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Im in the first layer        
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    2 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    7 Some Query String           10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Im in the first layer        
    7 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    7 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    7 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    7 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    7 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer
    7 2nd Layer Some Query String 10/19/2017 12:00:00 AM 10/19/2017 11:06:44 AM Something at the second layer

Thanks in advance for your help!

MVKozlov commented 6 years ago

I see there is a problem with $PoshRS_JobIDm it not incremented as it sould (bug!) because it replaced, other details is hidden...

but, pipelined object inside scriptblock shoud be $_ and you use $_ as parameter. that can broke parameter detection and (may be) cant be fixed

btw, monstrous (get-date).AddHours(-(get-date).Hour).AddMinutes(-(get-date).Minute).AddSeconds(-(get- it just (Get-Date).Date

magno101 commented 6 years ago

Thanks! I've modified it as such and it seems to work as intended! Appreciate the help, this helped solved a big issue for me.

`$Array = @(1..(Get-Random -Maximum 10 -Minimum 2)) $Query = "Some Query String" $Start = (Get-Date).Date $End = (get-date)

Write-Host "Query: $Query" Write-Host "Start: $Start" Write-Host "End : $End"

$Jobs = $Array | Start-RSJob -Name {"Layer1$"} -ScriptBlock { [pscustomobject]@{ Array = $_ Query = $Using:Query Start = $Using:Start End = $Using:End Debug = "Im in the first layer" } $NewArray = @(10..(Get-Random -Maximum 20 -Minimum 11))

$InnerJobs = $NewArray | Start-RSJob -ArgumentList $Using:Query,$Using:Start,$Using:End -Name {"Layer2$"} -ScriptBlock { param( $Query, $Start, $End ) [pscustomobject]@{ Array = $_ Query = "2nd Layer " + $Query Start = $Start End = $End Debug = "Something at the second layer" } } $InnerJobs | Wait-RSJob | Receive-RSJob }

$Jobs | Wait-RSJob | Receive-RSJob | ft -AutoSize`