Replacing foreach loop with RSJobs

[chrisdma]

Hello, I have an LDAP query that queries an AD Domain Controller via global catalog for all users that meet the search criteria, however it takes about 6 minutes to execute. I replaced it with a foreach loop of the specific OUs I need to search (that contain Users in the OU distinguishedName) and it took about 9 minutes still, as the loop is concurrent instead of parallel. I am looking through these examples and am stuck on trying to replace this foreach loop with parallel jobs, can someone assist at least in how to get started? My code with the foreach loop is as seen below:

I would assume that I would need to count the rows in $masterOuList (which is about 3000 OUs) and start a number of jobs equal to the count of the rows? And then I am not sure how to pipe that row's properties ($row.OU and $row.DC when using a foreach loop) into each job as its not a static variable. And on top of that, the ArgumentList and $using: parts are confusing me too, I just need some kind of direction to start.

Thanks,

Chris

[kensel]

I don't see where you are sending the Find-LdapObject output as it looks like you just send it out to output stream.

The start-RSJob passes objects from the pipeline in using same $_ as a foreach-object script block so something like the following should work:

$daysOld = (Get-Date).AddDays(-3).ToUniversalTime().ToString('yyyyMMddHHmmss.fZ') Register-LdapAttributeTransform securityDescriptor Import-Csv "C:\Temp\MasterOuList.csv" | Start-RSJob { Find-LdapObject -LdapConnection (get-LdapConnection -LdapServer $.DC -Port 3268) -SearchFilter:"(&(objectClass=user)(objectCateory=OrganizationalPerson)(whenCreated > $Using:daysOld)(!(emailaddress=*)))" -SearchBase:$.OU -PropertiesToLoad:@('sAMAccountName','whenCreated','emailAddress','extensionAttribute9','ntSecurityDescriptor') -AdditionalControls (New-Object System.DirectoryServices.Protocols.SecurityDescriptorFlagControl('Owner')) -BinaryProps:@('ntSecurityDescriptor') } Get-RSJob | Wait-RSJob | Receive-RSJob

I don't have experience with the LDAP commands your running. Let me know if the above works for you. Another option if you are more familiar with

Note: not sure if it is the preview messing with the code formatting but screenshot should be able to clear it up for the copy and paste.

[chrisdma]

I don't see where you are sending the Find-LdapObject output as it looks like you just send it out to output stream.

The start-RSJob passes objects from the pipeline in using same $_ as a foreach-object script block so something like the following should work:

$daysOld = (Get-Date).AddDays(-3).ToUniversalTime().ToString('yyyyMMddHHmmss.fZ') Register-LdapAttributeTransform securityDescriptor Import-Csv "C:\Temp\MasterOuList.csv" | Start-RSJob { Find-LdapObject -LdapConnection (get-LdapConnection -LdapServer $.DC -Port 3268) -SearchFilter:"(&(objectClass=user)(objectCateory=OrganizationalPerson)(whenCreated > $Using:daysOld)(!(emailaddress=*)))" -SearchBase:$.OU -PropertiesToLoad:@('sAMAccountName','whenCreated','emailAddress','extensionAttribute9','ntSecurityDescriptor') -AdditionalControls (New-Object System.DirectoryServices.Protocols.SecurityDescriptorFlagControl('Owner')) -BinaryProps:@('ntSecurityDescriptor') } Get-RSJob | Wait-RSJob | Receive-RSJob

I don't have experience with the LDAP commands your running. Let me know if the above works for you. Another option if you are more familiar with

Note: not sure if it is the preview messing with the code formatting but screenshot should be able to clear it up for the copy and paste.

Hey kendel,

Thanks so much for your assistance. I normally pipe the output of the Find-LdapObject to Out-GridView (however in the code I posted I was outputting to the standard output stream for testing purposes), so if your example works would it be possible to pipe Receive-RSJob to that as well? I assume Receive-RSJob would just receive the resulting objects from Find-LdapObject correct?

[kensel]

Correct, Receive-RSJob returns the results. One thing to note though, if you are running this in a session that has other RSJobs you would want to set a variable to receive the RSJob objects that are returned by the start-rsjob and then pipe them to 'Wait-RSJob | Receive-RSJob | Out-GridView' as the Get-RSJob would return all including previous runs that haven't been cleaned up with Remove-RSJob. Or change line 11 and 12 to be '} | Wait-RSJob | Receive-RSJob | Out-GridView'

[chrisdma]

Correct, Receive-RSJob returns the results. One thing to note though, if you are running this in a session that has other RSJobs you would want to set a variable to receive the RSJob objects that are returned by the start-rsjob and then pipe them to 'Wait-RSJob | Receive-RSJob | Out-GridView' as the Get-RSJob would return all including previous runs that haven't been cleaned up with Remove-RSJob. Or change line 11 and 12 to be '} | Wait-RSJob | Receive-RSJob | Out-GridView'

So this is what I am trying at the moment:

If I recall Get-RSJob after about a minute while waiting for all jobs to complete, all of the jobs' HasMoreData attribute is false, as in they aren't capturing the Find-LdapObject from the pipeline for some reason. The code completely works on its own without PoshRSJobs and I am confused why either the Find-LdapObject command isn't executing, or why the job can't capture the object. (As HasMoreData is false, piping Get-RSJob to Wait-RSJob to Receive-RSJob doesn't output anything at all.)

I tried doing this in a ScriptBlock as your example without a ScriptBlock didn't work either, but this one ended up not outputting anything as well. This is so frustrating lol.

[chrisdma]

And to make sure the pipeline DC, OU, and $daysOld variable were being passed into the RSJob correctly, I tested with a Write-Output and they are, as seen below. Blurred the contents as it gives away who I work for:

I originally thought this would be the issue but those variables are completely fine within the RSJob as I tried without "$using:" and just did $daysOld and got an error from Find-LdapObject saying the filter parameter was invalid. So with "$using:", the parameter is obviously correct and I'm sure it attempts to execute, I just don't know why the user objects aren't being passed to the pipeline.

[chrisdma]

Nix everything above, I'm a moron. A total damn moron lol. The SearchFilter in my recently provided screenshot shows a typo in it; Should be objectCategory, wow. Typed that one filter in manually when moving the Find-LdapObject process over to PoshRSJobs and totally screwed that up. nice. The PoshRSJobs module works perfectly now, thank you so much for your help @kensel.