A couple of suggestions

[JFBDO2022]

First item - for performance improvement, change line 958 from Write-Progress -Activity $activity -Status "$processedCount of $itemCount item(s) read" -PercentComplete (($processedCount/$itemCount)*100)

To if ($processedCount % 100 -eq 0) { Write-Progress -Activity $activity -Status "$processedCount of $itemCount item(s) read" -PercentComplete (($processedCount/$itemCount)*100) }

Reading in a lot of records and echoing that progress slowed down the discovery step quite a bit. This way, you get a bit better performance and a frequent-enough update on progress.

Second item - add a $MaximumToProcess parameter.

I cheated and just threw in a line 969: If ($processedCount -gt 100000) { $moreItems = $false }

But ideally, you could do that as a parameter, have it default to 250k or some number like that, then work that into the discovery process and throw a message "found more items but capped to 250000" and change my suggested 969 line to use the variable feed.

Doing smaller chunks like that improved some performance aspects (didn't seem quite linear in the scaling for how long things took - did 50k chunks while could, then kicked off a bigger batch when just wanted it to run for a long stretch).

Thanks.

[David-Barrett-MS]

Which script is this referring to?

[JFBDO2022]

Sorry - guess the page covers a few. Remove-DuplicateItems.ps1 Thanks.

[David-Barrett-MS]

v1.1.2 is published, and it supports a debugMaxItems variable (defined at the beginning of the script). I have added the logging optimisation too, but that will only appear in the next version of the script (I missed it for this release).