GSUser Array IndexOf not working with variables

IronKane2 commented 5 years ago

I load the variable $GAFEUsers from Get-GSUser

$GAFEUsers.PrimaryEmail.indexof("cs70079@djusdstudents.org")

Will return an index

$Email = ""cs70079@djusdstudents.org""
$GAFEUsers.PrimaryEmail.indexof($Email)

Returns -1

The full command is supposed to return a single user object

$GAFEUsr = $GAFEUsers[$GAFEUsers.PrimaryEmail.IndexOf($Email)]

I'm also doing the same kind of search on an array of ADUser objcects and have no problems. So I'm fairly confident that my syntax is correct. Both variables are of the type System.Array.

I know I could use the Where-Object

$GAFEUsr = $GAFEUsers | ?{$_.PrimaryEmail -eq $Email}

But I noticed that it was substantially slower even when the IndexOf version was not finding a user.

Any help would be appreciated.

Steve

scrthq commented 5 years ago

@IronKane2 - $Email = ""cs70079@djusdstudents.org"" << not sure if the double double-quotes was intentional since that isn't valid PowerShell by itself and won't store a string as expected. I'm not able to replicate the issue though; both passing the string directly as well as passing the variable with the string yield the same results:

$GAFEUsers = Get-GSUser -Filter * -Verbose
$GAFEUsers.PrimaryEmail.indexof("n@ferrell.io") # Returns 1 as expected
$Email = "n@ferrell.io"
$GAFEUsers.PrimaryEmail.indexof($Email) # Returns 1 as well
$Email = ""n@ferrell.io"" # Throws an Unexpected Token error since it's trying to evaluate my email as a command

scrthq commented 5 years ago

what I would strongly suggest for this though would actually be storing your users in a hashtable then performing your lookup against that:

$GAFEUsers = Get-GSUser -Filter * -Verbose
$userHash = @{}
foreach ($user in $GAFEUsers) {
    foreach ($address in $user.Emails.'Address') {
        $userHash[$address] = $user
    }
}
$userHash["cs70079@djusdstudents.org"] # Returns the user object
$Email = "cs70079@djusdstudents.org"
$userHash[$Email] # Returns the user object as well

For performance metrics, filling out that hashtable with over 10K user objects in my org takes 0.09 seconds and lookups against it are pretty much instant.

The performance gains going this route are actually significant enough to have a convenience wrapper for this already built into PSGSuite: Sync-GSUserCache

https://github.com/scrthq/PSGSuite/wiki/Sync-GSUserCache

Running Sync-GSUserCache with default parameters fills out a global variable called GSUserCache with a hashtable containing every email, alias and user ID in your G Suite account as keys.

Usage:

Sync-GSUserCache
$global:GSUserCache["cs70079@djusdstudents.org"] # Returns the user object
$Email = "cs70079@djusdstudents.org"
$global:GSUserCache[$Email] # Returns the user object as well

In case you don't want to worry about remembering that global variable, you can choose to PassThru the result to store in a local variable:

$hash = Sync-GSUserCache -PassThru
$hash["cs70079@djusdstudents.org"] # Returns the user object
$Email = "cs70079@djusdstudents.org"
$hash[$Email] # Returns the user object as well

scrthq commented 5 years ago

For any low number of matches (looking like anything sub-100 items to match), IndexOf beats a Hashtable lookup by a nose, but Hashtables really shine when having to do a large amount of lookups, showing how scalable it can be:

$allUsers = Get-GSUser -Filter * -Verbose

$results = @()
$start = Get-Date
foreach ($count in 10,20,50,100,500,1000) {
    $random = $allUsers.PrimaryEmail | Get-Random -Count $count

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$(((Get-Date) - $start).ToString())] [$count] Testing Where-Object performance"
    $res = Measure-Command {
        foreach ($email in $random) {
            $allUsers | Where-Object {$_.PrimaryEmail -eq $email}
        }
    }
    $results += [PSCustomObject]@{
        Method = 'Where-Object'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$(((Get-Date) - $start).ToString())] [$count] Testing IndexOf performance"
    $res = Measure-Command {
        foreach ($email in $random) {
            $allUsers[$allUsers.PrimaryEmail.IndexOf($email)]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'IndexOf'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$(((Get-Date) - $start).ToString())] [$count] Testing Hashtable performance"
    $res = Measure-Command {
        $userHash = @{}
        foreach ($user in $allUsers) {
            foreach ($address in $user.Emails.'Address') {
                $userHash[$address] = $user
            }
        }
        foreach ($email in $random) {
            $userHash[$email]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'Hashtable'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }
}
$results | Sort-Object Method,Count | Format-Table -AutoSize

Resulting output:

[13:07:48.8525 +00:00:00.1037216] [10] Testing Where-Object performance
[13:07:54.0067 +00:00:05.2579342] [10] Testing IndexOf performance
[13:07:54.0765 +00:00:05.3277468] [10] Testing Hashtable performance
[13:07:54.4355 +00:00:05.6867589] [20] Testing Where-Object performance
[13:08:05.3253 +00:00:16.5766034] [20] Testing IndexOf performance
[13:08:05.4151 +00:00:16.6663630] [20] Testing Hashtable performance
[13:08:05.7522 +00:00:17.0034875] [50] Testing Where-Object performance
[13:08:32.5455 +00:00:43.7967662] [50] Testing IndexOf performance
[13:08:32.7290 +00:00:43.9802809] [50] Testing Hashtable performance
[13:08:33.0262 +00:00:44.2784492] [100] Testing Where-Object performance
[13:09:26.8860 +00:01:38.1383102] [100] Testing IndexOf performance
[13:09:27.3189 +00:01:38.5701494] [100] Testing Hashtable performance
[13:09:27.6649 +00:01:38.9161964] [500] Testing Where-Object performance
[13:14:05.2061 +00:06:16.4573587] [500] Testing IndexOf performance
[13:14:06.8856 +00:06:18.1378910] [500] Testing Hashtable performance
[13:14:07.2427 +00:06:18.4939105] [1000] Testing Where-Object performance
[13:23:21.5029 +00:15:32.7541608] [1000] Testing IndexOf performance
[13:23:24.5767 +00:15:35.8279750] [1000] Testing Hashtable performance

Method       Count TotalSeconds
------       ----- ------------
Hashtable       10    0.2277754
Hashtable       20    0.2068724
Hashtable       50    0.1953845
Hashtable      100    0.2088513
Hashtable      500    0.2233988
Hashtable     1000    0.2406013
IndexOf         10    0.0673751
IndexOf         20    0.0875779
IndexOf         50    0.1814888
IndexOf        100    0.4300367
IndexOf        500    1.6769426
IndexOf       1000    3.0713863
Where-Object    10    5.1300292
Where-Object    20   10.8874049
Where-Object    50   26.7911417
Where-Object   100   53.8566273
Where-Object   500  277.5388551
Where-Object  1000  554.2566299

IronKane2 commented 5 years ago

Sorry to open this with a late reply, but I just wanted to say that you've opened my eyes to the utility of hash tables. Once I quit looking at it as array of string pairs and looked at it as a collection of objects...I was sold. I immediately converted my CSV and ADUser arrays to hash tables. It's worth the time to load them.

Thanks again. Steve

scrthq commented 5 years ago

That's awesome, thanks for sharing!!! It took me a while to realize the same thing tbh, it's insane how performant it is in comparison though!

_{Sent with GitHawk}

scrthq commented 5 years ago

@IronKane2 - If you want to cut it down even a bit further, you can use a Dictionary<string,object> instead:

Script:

Import-Module PSGSuite
$allUsers = Get-GSUser -Filter * -Verbose

$results = @()
$start = Get-Date
foreach ($count in 10,20,50,100,500,1000) {
    $random = $allUsers.PrimaryEmail | Get-Random -Count $count

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$(((Get-Date) - $start).ToString())] [$count] Testing Dictionary performance"
    $res = Measure-Command {
        $userDict = New-Object 'System.Collections.Generic.Dictionary[[System.String],[System.Object]]'
        foreach ($user in $allUsers) {
            foreach ($address in ($user.Emails.'Address')) {
                $userDict.Add($address,$user)
            }
        }
        foreach ($email in $random) {
            $userDict[$email]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'Dictionary'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$(((Get-Date) - $start).ToString())] [$count] Testing IndexOf performance"
    $res = Measure-Command {
        foreach ($email in $random) {
            $allUsers[$allUsers.PrimaryEmail.IndexOf($email)]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'IndexOf'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$(((Get-Date) - $start).ToString())] [$count] Testing Hashtable performance"
    $res = Measure-Command {
        $userHash = @{}
        foreach ($user in $allUsers) {
            foreach ($address in $user.Emails.'Address') {
                $userHash[$address] = $user
            }
        }
        foreach ($email in $random) {
            $userHash[$email]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'Hashtable'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }
}
$results | Sort-Object Method,Count | Format-Table -AutoSize

Results:

[00:48:25.8891 +00:00:00.0549697] [10] Testing Dictionary performance
[00:48:25.9601 +00:00:00.1259639] [10] Testing IndexOf performance
[00:48:25.9851 +00:00:00.1509649] [10] Testing Hashtable performance
[00:48:26.1541 +00:00:00.3199995] [20] Testing Dictionary performance
[00:48:26.2491 +00:00:00.4149670] [20] Testing IndexOf performance
[00:48:26.2991 +00:00:00.4650023] [20] Testing Hashtable performance
[00:48:26.4631 +00:00:00.6289645] [50] Testing Dictionary performance
[00:48:26.5541 +00:00:00.7199638] [50] Testing IndexOf performance
[00:48:26.6551 +00:00:00.8209651] [50] Testing Hashtable performance
[00:48:26.8081 +00:00:00.9740003] [100] Testing Dictionary performance
[00:48:26.8981 +00:00:01.0639633] [100] Testing IndexOf performance
[00:48:27.0681 +00:00:01.2340023] [100] Testing Hashtable performance
[00:48:27.2191 +00:00:01.3849998] [500] Testing Dictionary performance
[00:48:27.3101 +00:00:01.4759635] [500] Testing IndexOf performance
[00:48:28.1501 +00:00:02.3159961] [500] Testing Hashtable performance
[00:48:28.3131 +00:00:02.4789640] [1000] Testing Dictionary performance
[00:48:28.4031 +00:00:02.5690017] [1000] Testing IndexOf performance
[00:48:30.2431 +00:00:04.4089686] [1000] Testing Hashtable performance

Method     Count TotalSeconds
------     ----- ------------
Dictionary    10    0.0678016
Dictionary    20    0.0916306
Dictionary    50    0.0883446
Dictionary   100    0.0878136
Dictionary   500    0.0879438
Dictionary  1000    0.0876976
Hashtable     10    0.1193066
Hashtable     20    0.1042325
Hashtable     50    0.0967026
Hashtable    100    0.0974746
Hashtable    500    0.1074829
Hashtable   1000    0.1004324
IndexOf       10    0.0215172
IndexOf       20    0.0411156
IndexOf       50    0.0987539
IndexOf      100    0.1678052
IndexOf      500    0.8365078
IndexOf     1000    1.8379033

scrthq commented 5 years ago

Cranked up to 10K lookups:

[00:58:13.8793 +00:00:00.0510005] [10] Testing Dictionary performance
[00:58:13.9523 +00:00:00.1239980] [10] Testing IndexOf performance
[00:58:13.9733 +00:00:00.1450333] [10] Testing Hashtable performance
[00:58:14.1543 +00:00:00.3259996] [20] Testing Dictionary performance
[00:58:14.2393 +00:00:00.4109987] [20] Testing IndexOf performance
[00:58:14.2863 +00:00:00.4579998] [20] Testing Hashtable performance
[00:58:14.4423 +00:00:00.6139970] [50] Testing Dictionary performance
[00:58:14.5363 +00:00:00.7080045] [50] Testing IndexOf performance
[00:58:14.6413 +00:00:00.8130025] [50] Testing Hashtable performance
[00:58:14.7973 +00:00:00.9689974] [100] Testing Dictionary performance
[00:58:14.8933 +00:00:01.0650403] [100] Testing IndexOf performance
[00:58:15.1083 +00:00:01.2800430] [100] Testing Hashtable performance
[00:58:15.2614 +00:00:01.4330750] [500] Testing Dictionary performance
[00:58:15.3663 +00:00:01.5380409] [500] Testing IndexOf performance
[00:58:16.2444 +00:00:02.4160735] [500] Testing Hashtable performance
[00:58:16.4003 +00:00:02.5720405] [1000] Testing Dictionary performance
[00:58:16.4933 +00:00:02.6650420] [1000] Testing IndexOf performance
[00:58:18.8893 +00:00:05.0610428] [1000] Testing Hashtable performance
[00:58:19.0563 +00:00:05.2290409] [1500] Testing Dictionary performance
[00:58:19.1533 +00:00:05.3250457] [1500] Testing IndexOf performance
[00:58:22.0893 +00:00:08.2620460] [1500] Testing Hashtable performance
[00:58:22.2513 +00:00:08.4230455] [2000] Testing Dictionary performance
[00:58:22.3463 +00:00:08.5180397] [2000] Testing IndexOf performance
[00:58:25.8953 +00:00:12.0670417] [2000] Testing Hashtable performance
[00:58:26.0513 +00:00:12.2230433] [2500] Testing Dictionary performance
[00:58:26.1394 +00:00:12.3110742] [2500] Testing IndexOf performance
[00:58:30.1663 +00:00:16.3380444] [2500] Testing Hashtable performance
[00:58:30.3353 +00:00:16.5070445] [3000] Testing Dictionary performance
[00:58:30.4523 +00:00:16.6240420] [3000] Testing IndexOf performance
[00:58:35.1524 +00:00:21.3240747] [3000] Testing Hashtable performance
[00:58:35.3124 +00:00:21.4840745] [4000] Testing Dictionary performance
[00:58:35.4003 +00:00:21.5720436] [4000] Testing IndexOf performance
[00:58:42.2644 +00:00:28.4360777] [4000] Testing Hashtable performance
[00:58:42.4284 +00:00:28.6000784] [5000] Testing Dictionary performance
[00:58:42.5173 +00:00:28.6890422] [5000] Testing IndexOf performance
[00:58:50.6323 +00:00:36.8040442] [5000] Testing Hashtable performance
[00:58:50.8123 +00:00:36.9840460] [6000] Testing Dictionary performance
[00:58:50.9014 +00:00:37.0730724] [6000] Testing IndexOf performance
[00:59:00.8093 +00:00:46.9810476] [6000] Testing Hashtable performance
[00:59:00.9843 +00:00:47.1560473] [8000] Testing Dictionary performance
[00:59:01.0753 +00:00:47.2470489] [8000] Testing IndexOf performance
[00:59:15.0223 +00:01:01.1940501] [8000] Testing Hashtable performance
[00:59:15.2163 +00:01:01.3880498] [10000] Testing Dictionary performance
[00:59:15.3233 +00:01:01.4950462] [10000] Testing IndexOf performance
[00:59:34.1714 +00:01:20.3430784] [10000] Testing Hashtable performance

Method     Count TotalSeconds
------     ----- ------------
Dictionary    10    0.0700478
Dictionary    20    0.0835829
Dictionary    50    0.0916977
Dictionary   100    0.0925477
Dictionary   500    0.1021467
Dictionary  1000    0.0890968
Dictionary  1500    0.0937333
Dictionary  2000    0.0935613
Dictionary  2500    0.0845769
Dictionary  3000     0.114307
Dictionary  4000    0.0848459
Dictionary  5000    0.0864978
Dictionary  6000    0.0856764
Dictionary  8000    0.0876938
Dictionary 10000    0.1049154
Hashtable     10    0.1182719
Hashtable     20    0.0992421
Hashtable     50    0.1067307
Hashtable    100    0.1018507
Hashtable    500    0.0963409
Hashtable   1000    0.1111319
Hashtable   1500    0.0964067
Hashtable   2000    0.1006189
Hashtable   2500    0.1038251
Hashtable   3000    0.1009152
Hashtable   4000    0.1064309
Hashtable   5000    0.1195701
Hashtable   6000    0.1101727
Hashtable   8000    0.1285827
Hashtable  10000    0.1254782
IndexOf       10     0.018703
IndexOf       20    0.0437742
IndexOf       50    0.1022288
IndexOf      100    0.2118295
IndexOf      500    0.8747392
IndexOf     1000    2.3937085
IndexOf     1500    2.9334654
IndexOf     2000    3.5469876
IndexOf     2500    4.0249885
IndexOf     3000    4.6983745
IndexOf     4000    6.8604111
IndexOf     5000     8.111934
IndexOf     6000    9.9047336
IndexOf     8000   13.9443904
IndexOf    10000   18.8395156

scrthq commented 5 years ago

An even more verbose version... the most telling section is this:

DONE! Total time: 00:02:02.3810151

Total seconds per method over 43680 total lookups:
   - Dictionary: 2.3308223
   - IndexOf: 115.2580856
   - Hashtable: 3.192184

Full script:

Import-Module PSGSuite
$allUsers = Get-GSUser -Filter * -Verbose

$results = @()
$start = Get-Date
$lookups = @(10,20,50,100,500,1000,1500,2000,2500,3000,4000,5000,6000,8000,10000)
foreach ($count in $lookups) {
    $random = $allUsers.PrimaryEmail | Get-Random -Count $count

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$((Get-Date) - $start)] [$count] Testing Dictionary performance"
    $res = Measure-Command {
        $userDict = New-Object 'System.Collections.Generic.Dictionary[[System.String],[System.Object]]'
        foreach ($user in $allUsers) {
            foreach ($address in ($user.Emails.'Address')) {
                $userDict.Add($address,$user)
            }
        }
        foreach ($email in $random) {
            $userDict[$email]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'Dictionary'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$((Get-Date) - $start)] [$count] Testing IndexOf performance"
    $res = Measure-Command {
        foreach ($email in $random) {
            $allUsers[$allUsers.PrimaryEmail.IndexOf($email)]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'IndexOf'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }

    Write-Host "[$(Get-Date -Format 'HH:mm:ss.ffff') +$((Get-Date) - $start)] [$count] Testing Hashtable performance"
    $res = Measure-Command {
        $userHash = @{}
        foreach ($user in $allUsers) {
            foreach ($address in $user.Emails.'Address') {
                $userHash[$address] = $user
            }
        }
        foreach ($email in $random) {
            $userHash[$email]
        }
    }
    $results += [PSCustomObject]@{
        Method = 'Hashtable'
        Count = $count
        TotalSeconds = $res.TotalSeconds
    }
}

"
DONE! Total time: $((Get-Date) - $start)

Total seconds per method over $(($lookups | Measure-Object -Sum).Sum) total lookups:"
$results | Group-Object Method | ForEach-Object {
    "   - $($_.Name): $(($_.Group.TotalSeconds | Measure-Object -Sum).Sum)"
}

"`nSectional breakdown:"

$results | Sort-Object Method,Count | Format-Table Method,Count,@{
    N="TotalSeconds"
    E={
        $leftLength = 2
        $rightLength = 4
        $sp = ([String]$_.TotalSeconds) -split "\."
        $fPad = "0" * ($leftLength - $sp[0].Length)
        if ($sp[1].Length -lt $rightLength) {
            $sp[1] = $sp[1] + ("0" * ($rightLength - $sp[1].Length))
        }
        "$fPad$($sp[0]).$(($sp[1]).Substring(0,$rightLength))"
    }
} -AutoSize

Results:

[01:10:48.6621 +00:00:00.0960054] [10] Testing Dictionary performance
[01:10:48.7406 +00:00:00.1745951] [10] Testing IndexOf performance
[01:10:48.7626 +00:00:00.1965989] [10] Testing Hashtable performance
[01:10:48.9246 +00:00:00.3585984] [20] Testing Dictionary performance
[01:10:49.1216 +00:00:00.5555982] [20] Testing IndexOf performance
[01:10:49.2026 +00:00:00.6365999] [20] Testing Hashtable performance
[01:10:49.4677 +00:00:00.9016401] [50] Testing Dictionary performance
[01:10:49.6477 +00:00:01.0816066] [50] Testing IndexOf performance
[01:10:49.8227 +00:00:01.2566082] [50] Testing Hashtable performance
[01:10:50.1097 +00:00:01.5436133] [100] Testing Dictionary performance
[01:10:50.2847 +00:00:01.7186157] [100] Testing IndexOf performance
[01:10:50.5767 +00:00:02.0116515] [100] Testing Hashtable performance
[01:10:50.8767 +00:00:02.3106216] [500] Testing Dictionary performance
[01:10:51.0457 +00:00:02.4806685] [500] Testing IndexOf performance
[01:10:52.0727 +00:00:03.5066683] [500] Testing Hashtable performance
[01:10:52.2457 +00:00:03.6796342] [1000] Testing Dictionary performance
[01:10:52.4167 +00:00:03.8506385] [1000] Testing IndexOf performance
[01:10:54.9937 +00:00:06.4276990] [1000] Testing Hashtable performance
[01:10:55.2798 +00:00:06.7137054] [1500] Testing Dictionary performance
[01:10:55.4537 +00:00:06.8876766] [1500] Testing IndexOf performance
[01:10:59.7758 +00:00:11.2107231] [1500] Testing Hashtable performance
[01:11:00.0768 +00:00:11.5107265] [2000] Testing Dictionary performance
[01:11:00.2728 +00:00:11.7067314] [2000] Testing IndexOf performance
[01:11:05.1928 +00:00:16.6267861] [2000] Testing Hashtable performance
[01:11:05.3948 +00:00:16.8287878] [2500] Testing Dictionary performance
[01:11:05.5678 +00:00:17.0017921] [2500] Testing IndexOf performance
[01:11:12.9429 +00:00:24.3768808] [2500] Testing Hashtable performance
[01:11:13.2489 +00:00:24.6828793] [3000] Testing Dictionary performance
[01:11:13.4019 +00:00:24.8358855] [3000] Testing IndexOf performance
[01:11:21.3551 +00:00:32.7890028] [3000] Testing Hashtable performance
[01:11:21.6470 +00:00:33.0809809] [4000] Testing Dictionary performance
[01:11:21.7861 +00:00:33.2200078] [4000] Testing IndexOf performance
[01:11:32.3542 +00:00:43.7881032] [4000] Testing Hashtable performance
[01:11:33.3362 +00:00:44.7701120] [5000] Testing Dictionary performance
[01:11:33.5432 +00:00:44.9771166] [5000] Testing IndexOf performance
[01:11:47.5743 +00:00:59.0082816] [5000] Testing Hashtable performance
[01:11:47.9213 +00:00:59.3552838] [6000] Testing Dictionary performance
[01:11:48.0363 +00:00:59.4702829] [6000] Testing IndexOf performance
[01:12:03.0875 +00:01:14.5214565] [6000] Testing Hashtable performance
[01:12:03.3785 +00:01:14.8124628] [8000] Testing Dictionary performance
[01:12:03.5095 +00:01:14.9434649] [8000] Testing IndexOf performance
[01:12:24.5288 +00:01:35.9627080] [8000] Testing Hashtable performance
[01:12:24.7138 +00:01:36.1477072] [10000] Testing Dictionary performance
[01:12:24.8328 +00:01:36.2667443] [10000] Testing IndexOf performance
[01:12:50.7321 +00:02:02.1660089] [10000] Testing Hashtable performance

DONE! Total time: 00:02:02.3810151

Total seconds per method over 43680 total lookups:
   - Dictionary: 2.3308223
   - IndexOf: 115.2580856
   - Hashtable: 3.192184

Sectional breakdown:

Method     Count TotalSeconds
------     ----- ------------
Dictionary    10 00.7465
Dictionary    20 00.9380
Dictionary    50 00.7622
Dictionary   100 00.7096
Dictionary   500 00.6672
Dictionary  1000 00.6732
Dictionary  1500 00.7045
Dictionary  2000 00.9363
Dictionary  2500 00.6889
Dictionary  3000 00.5054
Dictionary  4000 00.3645
Dictionary  5000 00.0504
Dictionary  6000 00.1181
Dictionary  8000 00.2834
Dictionary 10000 00.1593
Hashtable     10 00.8201
Hashtable     20 00.6235
Hashtable     50 00.0050
Hashtable    100 00.1296
Hashtable    500 00.7326
Hashtable   1000 00.7943
Hashtable   1500 00.0482
Hashtable   2000 00.7795
Hashtable   2500 00.9591
Hashtable   3000 00.8614
Hashtable   4000 00.3664
Hashtable   5000 00.3646
Hashtable   6000 00.8474
Hashtable   8000 00.4717
Hashtable  10000 00.1178
IndexOf       10 00.1855
IndexOf       20 00.7753
IndexOf       50 00.7051
IndexOf      100 00.8821
IndexOf      500 01.2323
IndexOf     1000 02.7441
IndexOf     1500 04.1990
IndexOf     2000 04.1303
IndexOf     2500 07.7248
IndexOf     3000 07.4916
IndexOf     4000 10.6492
IndexOf     5000 14.2528
IndexOf     6000 15.4808
IndexOf     8000 21.1615
IndexOf    10000 25.9657

IronKane2 commented 5 years ago

Wow, that's more than I think I want to try to ingest. I'll stick to the hash. I'm writing scripts that others may have to follow when I get hit by that bus everyone's always talking about.

Hashtable Question: How do I search the hash by an attribute of my user objects?

Prior to creating any new users, I want to make sure I'm not creating a duplicate. Student naming syntax is 1st initial + last initial + ID + @domain. Fairly user friendly and absolutely unique...unless you have a name change in the SIS database and the script creates another account with the same student ID but different initials.

So I want to search my hash of AD users for the EmployeeID attribute to see if an account exists first.

scrthq commented 5 years ago

@IronKane2 - I would add in a key to your hashtable for each attribute you want to use to do lookups tbh. Something like this:

$lookupKeys = @('SamAccountName','EmployeeId','Mail','UserPrincipalName')
$adUserHash = @{}

"Pulling user list"
$allADUsers = Get-ADUser -Filter * -Property $lookupKeys

"Filling hashtable"
foreach ($user in $allADUsers) {
    foreach ($lookup in $lookupKeys) {
        if ($keyVal = ($user | Select-Object -ExpandProperty $lookup)) {
            $adUserHash[$keyVal] = $user
        }
    }
}

With the above, you can lookup any user by any of the keys in the $lookupKeys array (so long as the user has that attribute). If the user doesn't have the attribute, this section guards against NullArrayIndex exceptions:

if ($keyVal = ($user | Select-Object -ExpandProperty $lookup)) {
    $adUserHash[$keyVal] = $user
}

SCRT-HQ / PSGSuite

GSUser Array IndexOf not working with variables #154