search

microsoft / Microsoft365DSC

Manages, configures, extracts and monitors Microsoft 365 tenant configurations
https://aka.ms/M365DSC
MIT License
1.53k stars 472 forks source link

ReportFromConfiguration: Dictionary Error with Key being added #3176

Closed meggenberger closed 1 year ago

meggenberger commented 1 year ago

Details of the scenario you tried and the problem that is occurring

I export some AAD components with Export-M365DSCConfiguration which works find. Then in a second step I create HTML Reports from the extracted configs with New-M365DSCReportFromConfiguration with the previous exported config as base. I do this for 3 environments (subscriptions - DEV, TEST and PROD). Now 2 environments it works great but in one environment I get an error:

[error]Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'CIMInstance' Key being added: 'CIMInstance'"

Verbose logs showing the problem

This is the second part of the logs from Azure DevOps Agent. First part contains Group and Usernames from the export which works find but I will not be able to share publicly. This second part of the log is generated when executing the following command:

New-M365DSCReportFromConfiguration -Type HTML -ConfigurationPath "$exportDir\M365TenantConfig.ps1" -OutputPath $exportFileName -Debug -Verbose

2023-04-18T07:06:08.4851142Z ⌛ Export took {454 seconds} 2023-04-18T07:06:09.0332851Z ##[command] Export Filename: D:\a_temp\CurrentConfig_AAD_2023-4-18.html 2023-04-18T07:06:09.0749862Z Folder PATH listing for volume Temporary Storage 2023-04-18T07:06:09.0750312Z Volume serial number is 12B9-E8D2 2023-04-18T07:06:09.0750473Z D:\A_TEMP 2023-04-18T07:06:09.0756261Z 9c57b347-b577-4ea2-9518-23315916caaf.ps1 2023-04-18T07:06:09.0756953Z ConfigurationData.psd1 2023-04-18T07:06:09.0757435Z M365TenantConfig.ps1 2023-04-18T07:06:09.0758019Z
2023-04-18T07:06:09.0758339Z No subfolders exist 2023-04-18T07:06:09.0759098Z 2023-04-18T07:06:09.1958364Z DEBUG: 00:00:00.0000731 Calling New() : MethodName = 'ResolvePackageSources' 2023-04-18T07:06:09.1961305Z DEBUG: 00:00:00.0001441 ErrorAction: SilentlyContinue 2023-04-18T07:06:09.1965857Z DEBUG: 00:00:00.0001939 Name: PSGallery 2023-04-18T07:06:09.1970694Z DEBUG: 00:00:00.0002410 WarningAction: SilentlyContinue 2023-04-18T07:06:09.1972625Z DEBUG: 00:00:00.0011845 INVOKING PowerShell Fn Resolve-PackageSource with args that has length 0 2023-04-18T07:06:09.1978311Z DEBUG: 00:00:00.0019151 In PowerShellGet Provider - 'Resolve-PackageSource'. 2023-04-18T07:06:09.1987804Z DEBUG: 00:00:00.0091783 Yielding package source for PSGallery at location https://www.powershellgallery.com/api/v2 2023-04-18T07:06:09.1993302Z DEBUG: 00:00:00.0099225 Done calling powershell «Resolve-PackageSource» «PSModule» 2023-04-18T07:06:09.1999591Z DEBUG: 00:00:00.0131844 Calling New() : MethodName = 'GetDynamicOptions' 2023-04-18T07:06:09.2003706Z DEBUG: 00:00:00.0132412 ErrorAction: SilentlyContinue 2023-04-18T07:06:09.2007854Z DEBUG: 00:00:00.0132895 Name: PSGallery 2023-04-18T07:06:09.2011968Z DEBUG: 00:00:00.0133362 WarningAction: SilentlyContinue 2023-04-18T07:06:09.2020779Z DEBUG: 00:00:00.0139133 INVOKING PowerShell Fn Get-DynamicOptions with args Provider that has length 1 2023-04-18T07:06:09.2026534Z DEBUG: 00:00:00.0145513 In PowerShellGet Provider - 'Get-DynamicOptions'. 2023-04-18T07:06:09.2032040Z DEBUG: 00:00:00.0159034 Done calling powershell «Get-DynamicOptions» «PSModule» 2023-04-18T07:06:09.2038323Z DEBUG: 00:00:00.0189498 Calling New() : MethodName = 'GetDynamicOptions' 2023-04-18T07:06:09.2042331Z DEBUG: 00:00:00.0190054 ErrorAction: SilentlyContinue 2023-04-18T07:06:09.2046246Z DEBUG: 00:00:00.0190543 Name: PSGallery 2023-04-18T07:06:09.2050353Z DEBUG: 00:00:00.0191012 WarningAction: SilentlyContinue 2023-04-18T07:06:09.2059543Z DEBUG: 00:00:00.0196399 INVOKING PowerShell Fn Get-DynamicOptions with args Source that has length 1 2023-04-18T07:06:09.2065187Z DEBUG: 00:00:00.0202527 In PowerShellGet Provider - 'Get-DynamicOptions'. 2023-04-18T07:06:09.2070550Z DEBUG: 00:00:00.0219358 Done calling powershell «Get-DynamicOptions» «PSModule» 2023-04-18T07:06:09.2116696Z DEBUG: 00:00:00.0382427 Calling New() : MethodName = 'ResolvePackageSources' 2023-04-18T07:06:09.2124822Z DEBUG: 00:00:00.0393185 ProviderName: PowerShellGet 2023-04-18T07:06:09.2133800Z DEBUG: 00:00:00.0402009 ErrorAction: SilentlyContinue 2023-04-18T07:06:09.2141993Z DEBUG: 00:00:00.0410413 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString 2023-04-18T07:06:09.2149835Z DEBUG: 00:00:00.0418332 Name: PSGallery 2023-04-18T07:06:09.2157103Z DEBUG: 00:00:00.0426719 WarningAction: SilentlyContinue 2023-04-18T07:06:09.2167794Z DEBUG: 00:00:00.0435640 INVOKING PowerShell Fn Resolve-PackageSource with args that has length 0 2023-04-18T07:06:09.2177318Z DEBUG: 00:00:00.0447493 In PowerShellGet Provider - 'Resolve-PackageSource'. 2023-04-18T07:06:09.2204989Z DEBUG: 00:00:00.0470321 Yielding package source for PSGallery at location https://www.powershellgallery.com/api/v2 2023-04-18T07:06:09.2210669Z DEBUG: 00:00:00.0480622 Done calling powershell «Resolve-PackageSource» «PSModule» 2023-04-18T07:06:09.2248806Z DEBUG: 00:00:00.0515088 Calling New() : MethodName = 'ResolvePackageSources' 2023-04-18T07:06:09.2256898Z DEBUG: 00:00:00.0525215 ProviderName: PowerShellGet 2023-04-18T07:06:09.2264893Z DEBUG: 00:00:00.0533278 ErrorAction: SilentlyContinue 2023-04-18T07:06:09.2273344Z DEBUG: 00:00:00.0541691 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString 2023-04-18T07:06:09.2281646Z DEBUG: 00:00:00.0550065 Name: PSGallery 2023-04-18T07:06:09.2288213Z DEBUG: 00:00:00.0558365 WarningAction: SilentlyContinue 2023-04-18T07:06:09.2298343Z DEBUG: 00:00:00.0566439 INVOKING PowerShell Fn Resolve-PackageSource with args that has length 0 2023-04-18T07:06:09.2305571Z DEBUG: 00:00:00.0576066 In PowerShellGet Provider - 'Resolve-PackageSource'. 2023-04-18T07:06:09.2330536Z DEBUG: 00:00:00.0597651 Yielding package source for PSGallery at location https://www.powershellgallery.com/api/v2 2023-04-18T07:06:09.2342122Z DEBUG: 00:00:00.0608456 Done calling powershell «Resolve-PackageSource» «PSModule» 2023-04-18T07:06:09.2597877Z DEBUG: 00:00:00.0000566 Calling New() : MethodName = 'GetDynamicOptions' 2023-04-18T07:06:09.2601136Z DEBUG: 00:00:00.0001161 Name: Microsoft365DSC 2023-04-18T07:06:09.2610030Z DEBUG: 00:00:00.0006568 INVOKING PowerShell Fn Get-DynamicOptions with args Provider that has length 1 2023-04-18T07:06:09.2616125Z DEBUG: 00:00:00.0012734 In PowerShellGet Provider - 'Get-DynamicOptions'. 2023-04-18T07:06:09.2621545Z DEBUG: 00:00:00.0021698 Done calling powershell «Get-DynamicOptions» «PSModule» 2023-04-18T07:06:09.2627567Z DEBUG: 00:00:00.0051973 Calling New() : MethodName = 'GetDynamicOptions' 2023-04-18T07:06:09.2631831Z DEBUG: 00:00:00.0052546 Name: Microsoft365DSC 2023-04-18T07:06:09.2640548Z DEBUG: 00:00:00.0057992 INVOKING PowerShell Fn Get-DynamicOptions with args Source that has length 1 2023-04-18T07:06:09.2646107Z DEBUG: 00:00:00.0064196 In PowerShellGet Provider - 'Get-DynamicOptions'. 2023-04-18T07:06:09.2651685Z DEBUG: 00:00:00.0080414 Done calling powershell «Get-DynamicOptions» «PSModule» 2023-04-18T07:06:09.2657925Z DEBUG: 00:00:00.0110223 Calling New() : MethodName = 'GetDynamicOptions' 2023-04-18T07:06:09.2662007Z DEBUG: 00:00:00.0110779 Name: Microsoft365DSC 2023-04-18T07:06:09.2671196Z DEBUG: 00:00:00.0116070 INVOKING PowerShell Fn Get-DynamicOptions with args Package that has length 1 2023-04-18T07:06:09.2676945Z DEBUG: 00:00:00.0122041 In PowerShellGet Provider - 'Get-DynamicOptions'. 2023-04-18T07:06:09.2682389Z DEBUG: 00:00:00.0156623 Done calling powershell «Get-DynamicOptions» «PSModule» 2023-04-18T07:06:09.2692552Z DEBUG: 00:00:00.0260342 Calling SearchForPackages. Name='Microsoft365DSC' 2023-04-18T07:06:09.2707849Z DEBUG: 00:00:00.0270937 PackageProvider::FindPackage with name Microsoft365DSC 2023-04-18T07:06:09.2708711Z DEBUG: 00:00:00.0275310 Calling SearchForPackages After Select 1 2023-04-18T07:06:09.2736835Z DEBUG: 00:00:00.0299816 Calling New() : MethodName = 'FindPackage' 2023-04-18T07:06:09.2745485Z DEBUG: 00:00:00.0311167 ProviderName: PowerShellGet 2023-04-18T07:06:09.2753928Z DEBUG: 00:00:00.0319651 Type: Module 2023-04-18T07:06:09.2762409Z DEBUG: 00:00:00.0328047 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString 2023-04-18T07:06:09.2768920Z DEBUG: 00:00:00.0336338 Name: Microsoft365DSC 2023-04-18T07:06:09.2780510Z DEBUG: 00:00:00.0344963 INVOKING PowerShell Fn Find-Package with args System.String[], , , that has length 4 2023-04-18T07:06:09.2788104Z DEBUG: 00:00:00.0355094 In PowerShellGet Provider - 'Find-Package'. 2023-04-18T07:06:09.2795929Z DEBUG: 00:00:00.0361860 OPTION: ProviderName => PowerShellGet 2023-04-18T07:06:09.2803446Z DEBUG: 00:00:00.0369551 OPTION: Type => Module 2023-04-18T07:06:09.2811101Z DEBUG: 00:00:00.0377153 OPTION: MessageResolver => Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString 2023-04-18T07:06:09.2816929Z DEBUG: 00:00:00.0384745 OPTION: Name => Microsoft365DSC 2023-04-18T07:06:09.2933186Z DEBUG: 00:00:00.0499828 PackageProvider::FindPackage with name Microsoft365DSC 2023-04-18T07:06:09.2965442Z DEBUG: 00:00:00.0529832 Calling 'NuGet'::'FindPackage' - name='Microsoft365DSC', requiredVersion='',minimumVersion='', 2023-04-18T07:06:09.2965869Z maximumVersion='''. 2023-04-18T07:06:09.2975323Z DEBUG: 00:00:00.0541488 Iterating 'Microsoft365DSC'. 2023-04-18T07:06:09.2987203Z DEBUG: 00:00:00.0549370 There are '1' registered sources in 'NuGet' provider. 2023-04-18T07:06:09.2999990Z DEBUG: 00:00:00.0562738 Source 'https://www.powershellgallery.com/api/v2' is not one of the registered sources in 2023-04-18T07:06:09.3000459Z 'NuGet' provider. 2023-04-18T07:06:09.3010498Z DEBUG: 00:00:00.0575849 Source 'https://www.powershellgallery.com/api/v2' is validated. 2023-04-18T07:06:09.3020052Z DEBUG: 00:00:00.0585404 Calling 'NuGetRequest'::'GetPackageById', 'Microsoft365DSC'. 2023-04-18T07:06:09.3029838Z DEBUG: 00:00:00.0595390 Calling 'NuGetPackageRepository'::'FindPackagesById', 'Microsoft365DSC'. 2023-04-18T07:06:09.3039626Z DEBUG: 00:00:00.0604737 Calling 'NuGetPackageFeed2'::'FindPackage', 'Microsoft365DSC'. 2023-04-18T07:06:09.3049094Z DEBUG: 00:00:00.0614802 Calling 'NuGetClient'::'FindPackage'. 2023-04-18T07:06:09.3060426Z DEBUG: 00:00:00.0628335 Downloading 2023-04-18T07:06:09.3061311Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=0&$top=40'. 2023-04-18T07:06:09.7561849Z DEBUG: 00:00:00.5128844 Completed downloading 2023-04-18T07:06:09.7563178Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=0&$top=40'. 2023-04-18T07:06:09.7665051Z DEBUG: 00:00:00.5226475 Downloading 2023-04-18T07:06:09.7665671Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=80&$top=40'. 2023-04-18T07:06:09.7669123Z DEBUG: 00:00:00.5228243 Downloading 2023-04-18T07:06:09.7669874Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=40&$top=40'. 2023-04-18T07:06:09.7675272Z DEBUG: 00:00:00.5239225 Downloading 2023-04-18T07:06:09.7675584Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=120&$top=40'. 2023-04-18T07:06:09.7677870Z DEBUG: 00:00:00.5244822 Downloading 2023-04-18T07:06:09.7678706Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=160&$top=40'. 2023-04-18T07:06:10.2561791Z DEBUG: 00:00:01.0128711 Completed downloading 2023-04-18T07:06:10.2562950Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=80&$top=40'. 2023-04-18T07:06:10.2819815Z DEBUG: 00:00:01.0386693 Downloading 2023-04-18T07:06:10.2820534Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=0&$top=40'. 2023-04-18T07:06:10.4167659Z DEBUG: 00:00:01.1734503 Completed downloading 2023-04-18T07:06:10.4169477Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=160&$top=40'. 2023-04-18T07:06:10.4173308Z DEBUG: 00:00:01.1735554 Completed downloading 2023-04-18T07:06:10.4174663Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=0&$top=40'. 2023-04-18T07:06:10.4269441Z DEBUG: 00:00:01.1831334 Downloading 2023-04-18T07:06:10.4271249Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=40&$top=40'. 2023-04-18T07:06:10.4274135Z DEBUG: 00:00:01.1832722 Downloading 2023-04-18T07:06:10.4274859Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=80&$top=40'. 2023-04-18T07:06:10.4277457Z DEBUG: 00:00:01.1834395 Downloading 2023-04-18T07:06:10.4278032Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=120&$top=40'. 2023-04-18T07:06:10.4280316Z DEBUG: 00:00:01.1835830 Downloading 2023-04-18T07:06:10.4281057Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=160&$top=40'. 2023-04-18T07:06:10.5600837Z DEBUG: 00:00:01.3165985 Completed downloading 2023-04-18T07:06:10.5601619Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=40&$top=40'. 2023-04-18T07:06:10.5764775Z DEBUG: 00:00:01.3332365 Completed downloading 2023-04-18T07:06:10.5765723Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=80&$top=40'. 2023-04-18T07:06:10.6291563Z DEBUG: 00:00:01.3857183 Downloading 2023-04-18T07:06:10.6292113Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=200&$top=40'. 2023-04-18T07:06:10.7833425Z DEBUG: 00:00:01.5400763 Completed downloading 2023-04-18T07:06:10.7834548Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=40&$top=40'. 2023-04-18T07:06:10.8110907Z DEBUG: 00:00:01.5676259 Downloading 2023-04-18T07:06:10.8111488Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=240&$top=40'. 2023-04-18T07:06:11.0114785Z DEBUG: 00:00:01.7679670 Completed downloading 2023-04-18T07:06:11.0115333Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=120&$top=40'. 2023-04-18T07:06:11.0404678Z DEBUG: 00:00:01.7972050 Completed downloading 2023-04-18T07:06:11.0405678Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=120&$top=40'. 2023-04-18T07:06:11.0460947Z DEBUG: 00:00:01.8028855 Completed downloading 2023-04-18T07:06:11.0462523Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=200&$top=40'. 2023-04-18T07:06:11.0821593Z DEBUG: 00:00:01.8385802 Downloading 2023-04-18T07:06:11.0822134Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=280&$top=40'. 2023-04-18T07:06:11.0830274Z DEBUG: 00:00:01.8386837 '0' packages received in the last request. 2023-04-18T07:06:11.0914138Z DEBUG: 00:00:01.8481619 Completed downloading 2023-04-18T07:06:11.0915200Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=160&$top=40'. 2023-04-18T07:06:11.1183849Z DEBUG: 00:00:01.8750271 '24' packages received in the last request. 2023-04-18T07:06:11.1277128Z DEBUG: 00:00:01.8842458 Completed downloading 2023-04-18T07:06:11.1277729Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=240&$top=40'. 2023-04-18T07:06:11.1291464Z DEBUG: 00:00:01.8854000 '0' packages received in the last request. 2023-04-18T07:06:11.1706385Z DEBUG: 00:00:01.9273877 Completed downloading 2023-04-18T07:06:11.1706788Z 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Microsoft365DSC'&$skip=280&$top=40'. 2023-04-18T07:06:11.1717292Z DEBUG: 00:00:01.9282004 '0' packages received in the last request. 2023-04-18T07:06:11.1729727Z DEBUG: 00:00:01.9295836 Completed iterating for 'Microsoft365DSC'. 2023-04-18T07:06:11.1871710Z DEBUG: 00:00:01.9433628 Done calling powershell «Find-Package» «PSModule» 2023-04-18T07:06:11.2703519Z VERBOSE: Loading file 'D:\a_temp\M365TenantConfig.ps1' 2023-04-18T07:06:13.6926614Z ##[error]Exception calling "Add" with "2" argument(s): "Item has already been added. Key in dictionary: 'CIMInstance' Key being added: 'CIMInstance'" 2023-04-18T07:06:13.9386674Z ##[error]PowerShell exited with code '1'. 2023-04-18T07:06:14.7833626Z ##[section]Finishing: Export the current config

Suggested solution to the issue

The DSC configuration that is used to reproduce the issue (as detailed as possible)

N/A

The operating system the target node is running

Microsoft Azure DevOps Windows Public Agent. Currently:

@{OsName=; OsOperatingSystemSKU=; OsArchitecture=; WindowsVersion=1809; WindowsBuildLabEx=17763.1.amd64fre.rs5_release.180914-1434; OsLanguage=; OsMuiLanguages=} -Message PSVersion: 5.1.17763.3770 -Message PSEdition: Desktop -Message Compatible Versions: -Message 1.0 2.0 3.0 4.0 5.0 5.1.17763.3770

Version of the DSC module that was used ('dev' if using current dev branch)

I'm using 

Install-Module -Name Microsoft365DSC -Force

Installing remote Modules - get-ComplianceReporting.ps1 Installing module Microsoft365DSC ... Installing module PnP.Powershell ... Installing DSCParser version {1.3.0.6} Installing ExchangeOnlineManagement version {3.1.0} Installing Microsoft.PowerApps.Administration.PowerShell version {2.0.159} WARNING: The names of some imported commands from the module 'Microsoft.PowerApps.AuthModule' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. WARNING: The names of some imported commands from the module 'Microsoft.PowerApps.Administration.PowerShell' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. Installing MicrosoftTeams version {5.1.0} Installing MSCloudLoginAssistant version {1.0.105} Installing ReverseDSC version {2.0.0.14} Installing renmote Modules - get-ComplianceReporting.ps1 All modules installed and loaded ✅

ModuleType Version Name ExportedCommands

Manifest 1.23.412.1 Microsoft365DSC {Add-M365DSCEvent, Add-M365DSCTelemetryEvent, Assert-M365D...

andikrueger commented 1 year ago

Any chance to share the configuration that causes issue? Please make sure to remove any sensitive information.

meggenberger commented 1 year ago

Let me try to reproduce the error with a smaller config. Currently I have a couple houndred group names and policies with names I would need to anonymise. I'll come back after the test.

meggenberger commented 1 year ago

Sry. I was out sick for a while. In the meantime there were some updates on the module and some cleanup on our side on the tenant. I suspect some escaping issue as we had the problem on one tenant only (all others worked) and the one not working was the testing tenant with all kinds of funny names. So it works now. Thanks anyways for your help.