microsoft / Microsoft365DSC

Manages, configures, extracts and monitors Microsoft 365 tenant configurations
https://aka.ms/M365DSC
MIT License
1.56k stars 486 forks source link

Export: Unable to create reports from configuration exports #4779

Open ChrisFisherAllianceIT opened 3 months ago

ChrisFisherAllianceIT commented 3 months ago

Description of the issue

When trying to create any type of report from an exported configuration. I receive the following error messages and it never completes. I have had this issue for a while now and I can confirm it was present in the last and current versions of the module. I just updated to version 1.24.619.1. Then I ran a new export and then tried to create a report from it. I even let it run for 4 days on the previous release where it eventually consumed all of my physical memory and 60GB of swap space before I gave up and killed the process.

image

Microsoft 365 DSC Version

Master / 1.24.619.1

Which workloads are affected

other

The DSC configuration

New-M365DSCReportFromConfiguration -Type HTML -ConfigurationPath "c:/path/config.ps1" -outputpath "c:/path/file.html"

New-M365DSCReportFromConfiguration -Type JSON -ConfigurationPath "c:/path/config.ps1" -outputpath "c:/path/file.json"

Verbose logs showing the problem

Invoke-Expression : At line:1 char:63
+ ...                                        $typeStaticMethods = [] | gm - ...
+                                                                  ~
Missing type name after '['.
At line:4 char:46
+                                             []::TryParse($subExpressi ...
+                                              ~
Missing type name after '['.
At C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.5\Modules\DSCParser.psm1:280 char:25
+ ...                       Invoke-Expression -Command $scriptBlock | Out-N ...
+                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ParserError: (:) [Invoke-Expression], ParseException
    + FullyQualifiedErrorId : MissingTypename,Microsoft.PowerShell.Commands.InvokeExpressionCommand

Environment Information + PowerShell Version

Name                           Value
----                           -----
PSVersion                      5.1.22621.3672
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.22621.3672
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
Mohit-Shivhare commented 3 months ago

I am also running similar commands to create reports from DSC config and getting below error. I updated Microsoft365DSC module and dependencies too. The current version is - 1.24.619.1.

Parsed content was null. At C:\Users\mshivhar\Documents\WindowsPowerShell\Modules\Microsoft365DSC\1.24.619.1\Modules\M365DSCReport.psm1:636 char:9

ChrisFisherAllianceIT commented 3 months ago

This issue is still present after updating to the latest release 1.24.626.1

AlexDiNicola commented 3 months ago

I have a similar error with Microsoft365DSC version 1.24.626.1 and DSCParser version 2.0.0.5. The error is the following:

Cannot index into a null array.
At C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.5\Modules\DSCParser.psm1:457 char:9
+         $resourceInstanceName = $resource.CommandElements[1].Value
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

And I saw that there are 3 lines where there are the same error (470, 456, 457):

Cannot index into a null array.                                                                                         At C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.5\Modules\DSCParser.psm1:**470** char:35                      + ... foreach ($keyValuePair in $resource.CommandElements[2].KeyValuePairs)                                             +                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                  + CategoryInfo          : InvalidOperation: (:) [], RuntimeException                                                    + FullyQualifiedErrorId : NullArray

Cannot index into a null array.
At C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.5\Modules\DSCParser.psm1:**456** char:9
+         $resourceType         = $resource.CommandElements[0].Value
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

Cannot index into a null array.
At C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.5\Modules\DSCParser.psm1:**457** char:9
+         $resourceInstanceName = $resource.CommandElements[1].Value
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

I have this error since the update to the version 2.0.0.5 of DSCParser. With the 2.0.0.4 version of DSCParser and the version 1.24.522.1 of M365DSC, I had some error but I did not have a null html output at the end of the New-M365DSCReportFromConfiguration

ricmestre commented 3 months ago

A fix for something unrelated was added to DSCParser 2.0.0.5 which now causes this problem, the workaround while that is not fixed is to just add a comment at the very first line of the blueprint as per below:

# This is the first line of my blueprint
Configuration ...
ChrisFisherAllianceIT commented 2 months ago

This work around didn't fix the issue. Some where it's not sending over or creating a useable $associatedCIMProperty.CIMType

HennepinCrawler commented 2 months ago

Same error, I swear this product is like a house of cards.

FabienTschanz commented 2 months ago

@Mohit-Shivhare Your issue should be fixed by #4867, that was reported outside of this issue. @ChrisFisherAllianceIT The type problem comes from Microsoft365DSC being unable to temporarily instantiate the CIM type, which requires administrative privileges. Please run the report creation from an administrative PowerShell session and check if the error is still present. @AlexDiNicola Can you provide a configuration for your issue and the version?

ChrisFisherAllianceIT commented 2 months ago

I have been running it in an Administrative Terminal which is where the issue first appeared. I have tried running it in an Administrative PowerShell session, I have tried different machines. The error is the same regardless of where or how I run it.

FabienTschanz commented 2 months ago

@ChrisFisherAllianceIT Can you provide a minimal failing configuration so that I can have a look at it? What's the output of Get-Module Microsoft365DSC -ListAvailable?

ChrisFisherAllianceIT commented 2 months ago

Unfortunately, I can't provide a config at this time, but I've had this issue no matter what tenant I run this against.

ModuleType Version Name ExportedCommands


Manifest 1.24.710.3 Microsoft365DSC {Assert-M365DSCBlueprint, Compare-M365DSCConfigurations, C...

FabienTschanz commented 2 months ago

I do have an idea. Maybe the size of the configuration you can send to the DSC provider is too small. In the registry, can you set the DWORD property maxEnvelopeSize under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WSMAN\Client and set its value to 8192? Afterwards, check that the configuration was applied successfully with Get-Item -Path WSMan:\localhost\MaxEnvelopeSizeKb, and then try it again.

Edit: If that doesn't work, can you export the contents of $Error after running the command in a fresh PowerShell session?

ChrisFisherAllianceIT commented 2 months ago

So that's interesting. There seem to be two issues that are missing from the installation instructions.

  1. WinRM wasn't enabled on my device.
  2. The Envelope size wasn't set.

Once I enabled WinRM and set the envelope size it fixed that error. Now I'm seeing a "Cannot index into a null array" for a few things but at least it's progressing

FabienTschanz commented 2 months ago

<Ignore comment please, didn't read carefully enough...>

Numb3rten commented 2 months ago

So that's interesting. There seem to be two issues that are missing from the installation instructions.

  1. WinRM wasn't enabled on my device.
  2. The Envelope size wasn't set.

Once I enabled WinRM and set the envelope size it fixed that error. Now I'm seeing a "Cannot index into a null array" for a few things but at least it's progressing

We have the same issue with error "Cannot index into a null array", with info "C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:513" "C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:514" "C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:527"

these are the lines in the DSCParser for the ressources: $resourceType = $resource.CommandElements[0].Value $resourceInstanceName = $resource.CommandElements[1].Value foreach ($keyValuePair in $resource.CommandElements[2].KeyValuePairs)

Do you have any idea, what could resolve this problem?

FabienTschanz commented 2 months ago

@Numb3rten and @ChrisFisherAllianceIT if possible, can you provide any sample configuration that throws the error and if possible an export of the $Error variable? Then we can have a look at the callstack and determine where the error occurred.

ricmestre commented 2 months ago

Most likely they have the same problem as myself, somewhere in the blueprint there's a string with configuration inside along with the Configuration initial line and the parser just can't cope with it.

As workaround for now just add an additional initial line as comment like this:

# Workaround
Configuration Example
{
...
}
Numb3rten commented 2 months ago

@Numb3rten and @ChrisFisherAllianceIT if possible, can you provide any sample configuration that throws the error and if possible an export of the $Error variable? Then we can have a look at the callstack and determine where the error occurred.

Sure:

Configuration MasterConfig
{
    param ()

    $OrganizationName = $ConfigurationData.NonNodeData.OrganizationName

    Import-DscResource -ModuleName 'Microsoft365DSC'

    Node localhost
    {

        AADAuthorizationPolicy c21ad7d9-4cef-4baf-81c6-2522c9cc482f
        {
            AllowedToSignUpEmailBasedSubscriptions                  = $False;
            AllowedToUseSSPR                                        = $True;
            AllowEmailVerifiedUsersToJoinOrganization               = $False;
            AllowInvitesFrom                                        = "adminsAndGuestInviters";
            BlockMsolPowerShell                                     = $True;
            ApplicationId                                           = $ConfigurationData.NonNodeData.ApplicationId;
            CertificateThumbprint                                   = $ConfigurationData.NonNodeData.CertificateThumbprint;
            TenantId                                                = $ConfigurationData.NonNodeData.TenantId;
            DefaultUserRoleAllowedToCreateApps                      = $False;
            DefaultUserRoleAllowedToCreateSecurityGroups            = $False;
            DefaultUserRoleAllowedToReadOtherUsers                  = $True;
            DefaultUserRoleAllowedToCreateTenants                   = $False;
            DefaultUserRoleAllowedToReadBitlockerKeysForOwnedDevice = $True;
            Description                                             = "Used to manage authorization related settings across the company.";
            DisplayName                                             = "Authorization Policy";
            Ensure                                                  = "Present";
            GuestUserRole                                           = "Guest";
            IsSingleInstance                                        = "Yes";
        }
        AADSecurityDefaults 501e1ed1-74b8-48f9-878b-d4ee0271d94c
        {
            ApplicationId             = $ConfigurationData.NonNodeData.ApplicationId;
            CertificateThumbprint     = $ConfigurationData.NonNodeData.CertificateThumbprint;
            TenantId                  = $ConfigurationData.NonNodeData.TenantId;
            DisplayName          = "Security Defaults";
            Description          = "Security defaults is a set of basic identity security mechanisms recommended by Microsoft. When enabled, these recommendations will be automatically enforced in your organization. Administrators and users will be better protected from common identity related attacks.";
            IsEnabled            = $False;
            IsSingleInstance     = "Yes";
        }
        AADTenantDetails bbeb241a-8c20-483d-918a-af38bf27914b
        {
            ApplicationId             = $ConfigurationData.NonNodeData.ApplicationId;
            CertificateThumbprint     = $ConfigurationData.NonNodeData.CertificateThumbprint;
            TenantId                  = $ConfigurationData.NonNodeData.TenantId;
            IsSingleInstance                     = "Yes";
            MarketingNotificationEmails          = @();
            SecurityComplianceNotificationMails  = @();
            SecurityComplianceNotificationPhones = @();
            TechnicalNotificationMails           = @($ConfigurationData.NonNodeData.TechnicalNotificationMails);
        }
        AADGroup 43f13936-21ab-4cbd-9d78-2ffaf96eb1bf
        {
            ApplicationId         = $ConfigurationData.NonNodeData.ApplicationId;
            CertificateThumbprint = $ConfigurationData.NonNodeData.CertificateThumbprint;
            DisplayName           = "SEC-M-ROL-ConditionalAccessEmergencyExcluded";
            Ensure                = "Present";
            GroupTypes            = @();
            MailEnabled           = $False;
            MailNickname          = "72bafaba-3";
            MemberOf              = @();
            Members               = $ConfigurationData.NonNodeData.ConditionalAccessEmergencyExcluded;
            Owners                = @();
            SecurityEnabled       = $True;
            TenantId              = $ConfigurationData.NonNodeData.TenantId;
        }
        #New Conditional Access Policy
        AADConditionalAccessPolicy "AADConditionalAccessPolicy-103 - Admin protection - AADP2 All apps: Block access For internal admins When any sign-in risk is detected"
        {
            ApplicationEnforcedRestrictionsIsEnabled = $False;
            ApplicationId                            = $ConfigurationData.NonNodeData.ApplicationId;
            BuiltInControls                          = @("block");
            CertificateThumbprint                    = $ConfigurationData.NonNodeData.CertificateThumbprint;
            ClientAppTypes                           = @("browser","mobileAppsAndDesktopClients");
            CloudAppSecurityIsEnabled                = $False;
            CloudAppSecurityType                     = "";
            CustomAuthenticationFactors              = @();
            DeviceFilterRule                         = "";
            DisplayName                              = "103 - Admin protection - AADP2 All apps: Block access For internal admins When any sign-in risk is detected";
            Ensure                                   = "Present";
            ExcludeApplications                      = @();
            ExcludeExternalTenantsMembers            = @();
            ExcludeGroups                            = @("SEC-M-ROL-ConditionalAccessEmergencyExcluded");
            ExcludeLocations                         = @();
            ExcludePlatforms                         = @();
            ExcludeRoles                             = @();
            ExcludeUsers                             = @();
            GrantControlOperator                     = "OR";
            Id                                       = "0286b6a1-3898-4f83-8de9-f431c02d46d3";
            IncludeApplications                      = @("All");
            IncludeExternalTenantsMembers            = @();
            IncludeGroups                            = @("SEC-M-ROL-ConditionalAccessAllAdmins");
            IncludeLocations                         = @();
            IncludePlatforms                         = @();
            IncludeRoles                             = @("Application Administrator","Application Developer","Attack Payload Author","Attack Simulation Administrator","Attribute Assignment Administrator","Attribute Assignment Reader","Attribute Definition Administrator","Attribute Definition Reader","Authentication Administrator","Authentication Policy Administrator","Azure DevOps Administrator","Azure Information Protection Administrator","Azure AD Joined Device Local Administrator","B2C IEF Policy Administrator","B2C IEF Keyset Administrator","Billing Administrator","Cloud App Security Administrator","Cloud Application Administrator","Cloud Device Administrator","Compliance Administrator","Compliance Data Administrator","Conditional Access Administrator","Customer LockBox Access Approver","Desktop Analytics Administrator","Directory Readers","Directory Synchronization Accounts","Directory Writers","Domain Name Administrator","Edge Administrator","Exchange Administrator","Dynamics 365 Administrator","Exchange Recipient Administrator","External ID User Flow Administrator","External ID User Flow Attribute Administrator","External Identity Provider Administrator","Global Administrator","Global Reader","Groups Administrator","Guest Inviter","Helpdesk Administrator","Hybrid Identity Administrator","Identity Governance Administrator","Insights Administrator","Insights Business Leader","Intune Administrator","Kaizala Administrator","Knowledge Administrator","Knowledge Manager","License Administrator","Message Center Privacy Reader","Message Center Reader","Network Administrator","Office Apps Administrator","Password Administrator","Fabric Administrator","Power Platform Administrator","Printer Administrator","Privileged Authentication Administrator","Printer Technician","Privileged Role Administrator","Search Administrator","Reports Reader","Search Editor","Security Administrator","Security Operator","Security Reader","Service Support Administrator","SharePoint Administrator","Skype for Business Administrator","Teams Administrator","Teams Communications Administrator","Teams Communications Support Engineer","Teams Communications Support Specialist","Teams Devices Administrator","Usage Summary Reports Reader","User Administrator","Windows 365 Administrator","Windows Update Deployment Administrator");
            IncludeUserActions                       = @();
            IncludeUsers                             = @();
            PersistentBrowserIsEnabled               = $False;
            PersistentBrowserMode                    = "";
            SignInFrequencyIsEnabled                 = $False;
            SignInFrequencyType                      = "";
            SignInRiskLevels                         = @("high","medium","low");
            State                                    = $ConfigurationData.NonNodeData.State103;
            TenantId                                 = $ConfigurationData.NonNodeData.TenantId;
            UserRiskLevels                           = @();
        }

This is a snipped of the bluepint and this is part of the $error:

Es ist nicht möglich, einen Index auf ein NULL-Array anzuwenden. 
In C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:527 Zeichen:35
+ ... foreach ($keyValuePair in $resource.CommandElements[2].KeyValuePairs)  
+                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~       
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException                                                                                                  
+ FullyQualifiedErrorId : NullArray       

Es ist nicht möglich, einen Index auf ein NULL-Array anzuwenden.
In C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:514 Zeichen:9
+         $resourceInstanceName = $resource.CommandElements[1].Value
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

Es ist nicht möglich, einen Index auf ein NULL-Array anzuwenden.
In C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:513 Zeichen:9
+         $resourceType         = $resource.CommandElements[0].Value
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

Es ist nicht möglich, einen Index auf ein NULL-Array anzuwenden.
In C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:527 Zeichen:35
+ ... foreach ($keyValuePair in $resource.CommandElements[2].KeyValuePairs)
+                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

Es ist nicht möglich, einen Index auf ein NULL-Array anzuwenden.
In C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:514 Zeichen:9
+         $resourceInstanceName = $resource.CommandElements[1].Value
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

Es ist nicht möglich, einen Index auf ein NULL-Array anzuwenden.
In C:\Program Files\WindowsPowerShell\Modules\DSCParser\2.0.0.7\Modules\DSCParser.psm1:513 Zeichen:9
+         $resourceType         = $resource.CommandElements[0].Value
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : NullArray

I hope it helps.

Most likely they have the same problem as myself, somewhere in the blueprint there's a string with configuration inside along with the Configuration initial line and the parser just can't cope with it.

As workaround for now just add an additional initial line as comment like this:

# Workaround
Configuration Example
{
...
}

I tried this, but no success.

Thank you very much

warrenstark commented 4 weeks ago

Tagging myself as impacted by this, happy to assist if any info is required to debug or test.

FabienTschanz commented 4 weeks ago

@warrenstark Can you share a minimal configuration set and the export of the $Error variable after a failed run? Please export the content of $Error to a file and upload it here, that helps us the best.

warrenstark commented 3 weeks ago

@FabienTschanz please find a minimal sharepoint export attached and error report.

error.txt M365TenantConfig.ps1.txt

Attempted to create a report from a config file using New-M365DSCReportFromConfiguration -Type html -ConfigurationPath .\M365TenantConfig.ps1 -OutputPath c:\config.html

FabienTschanz commented 3 weeks ago

@warrenstark Thank you very much for the configuration. I prepared a script that does pretty much the same as the logic of New-M365DSCReportFromConfiguration, but stripped down to the core functionality that is failing on your side.

Can you please run the script and upload the content? It does some verbose statements, so it's best to run it in a Windows PowerShell session that is fullscreen, so that you can then copy and paste it into a text file.

In the first line, you can change the file location of your M365TenantConfig.ps1. Then, just run it like .\Test-ConvertToDSCObject.ps1.

Test-ConvertToDSCObject.ps1.txt

warrenstark commented 3 weeks ago

Thanks @FabienTschanz , I've uploaded screen output and powershell transcript of running the PS1 above.

screen output.txt PowerShell_transcript.DESKTOP-O5FHP3R.tBKovw9Y.20240911073258.txt

FabienTschanz commented 3 weeks ago

@warrenstark I had a look at the recordings, and I had a hard time finding some differences (but I found some). Although that doesn't really make it easier, but we can now step deeper.

There seems to be an issue how the PowerShell Language parser goes through the content. While it works on my machine, I can see that for the actual resources (that should be one for the SPOTenantSettings and one for SPOHomeSite) there are four entries instead of two.

I have to be a bit more specific here: Normally, the parsed configuration contains one resource instance for each resource that is defined in the M365TenantConfig.ps1 file. This means that the instance for SPOHomeSite contains a type SPOHomeSite, a name (also SPOHomeSite according to the configuration) and then a hashtable with all the values (the actual config). In your case, the hashtable with the properties is separate from the type and name, resulting in an error as soon as the second element (only the hashtable) is run in the loop.

Unfortunately, I don't know why that is happening, so I want to do some more checks. I have prepared another script: Please run that one as well just as you did with the previous one. Test-ConvertToDSCObjectV2.ps1.txt

Edit: I have hardcoded the configuration into the script. If that works, then there must be something fishy going on with your file (although I checked your uploaded file and it is exactly the same as mine, even regarding carriage return and line feed).

warrenstark commented 3 weeks ago

Thanks @FabienTschanz have run this and uploaded here the output and transcript PowerShell_transcript.DESKTOP-O5FHP3R.+RKCNVeX.20240912084305.txt Test-ConvertToDSCObjectV2.ps1.output.txt

warrenstark commented 3 weeks ago

I'm definitely a noob in this space but I can see the output from the part in the try/catch block is not being quite as expected. If I pretend it throws an error and just runs the catch part, the rest of the test script seems to run with no errors.

Manually running the bit after running the "catch" part output is attached too, in case it helps [Uploading after running the catch part.txt…]()

FabienTschanz commented 3 weeks ago

@warrenstark Thank you very much for being my test subject at the moment. It's really difficult to debug your case 😢

I have now a third script with an improved detection which is based on fixed indices. Can you give that a try please? Test-ConvertToDSCObjectV3.ps1.txt

On another note, it seems like your last file (after running the catch part) didn't upload correctly.

warrenstark commented 3 weeks ago

@FabienTschanz no problems at all being a test bunny and agree, this one is certainly being difficult

Transcript of v3 and screenoutput attached here

v3_screenoutput.txt PowerShell_transcript.DESKTOP-O5FHP3R.ULSd2zCa.20240913081059.txt

FabienTschanz commented 3 weeks ago

@warrenstark Thanks a lot for your continued help. I think we might be on the right path, just trying to figure out how I can detect your situation. Please find attached the next test script: Test-ConvertToDSCObjectV4.ps1.txt

A bit more on my thoughts: I'm trying to figure out how we can check if the AST properly parsed the configuration and how we could fix it if it didn't.

warrenstark commented 2 weeks ago

V4 is run @FabienTschanz , thank you for your time looking into this! PowerShell_transcript.DESKTOP-O5FHP3R.b1GezDWD.20240916082436.txt v4output.txt

FabienTschanz commented 2 weeks ago

Thank you very much @warrenstark. Now I‘m seeing what the PowerShell parser sees, but not why though 😆 That puzzle still needs some solving, but we‘re on the right track.

In the meantime, could you do me a favor and run the script in PowerShell 7 once? Maybe there will be a difference. Just curious though, will need to be adressed for Windows PowerShell.

warrenstark commented 2 weeks ago

Thanks @FabienTschanz , same output in PS7 too.

v4output_inps7.txt PowerShell_transcript.DESKTOP-O5FHP3R.wAr2LMAD.20240916154455.txt

FabienTschanz commented 2 weeks ago

@warrenstark Silly question: What is the output of Get-Module Microsoft365DSC -ListAvailable? And could you also share the output of Get-DscResource -Module Microsoft365DSC -Verbose?

I found a crucial difference, and that is that on my machine, when running the parser, it automatically loads the Microsoft365DSC module and imports the DSC resources associated with it, thus enabling the detection of those resources inside the Node localhost block. If that is not done, it will behave the same way as it does on your machine. So I would like to find the difference in how the modules are installed and loaded.

warrenstark commented 2 weeks ago

PS C:\Windows\system32> Get-Module Microsoft365DSC -ListAvailable

Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version Name ExportedCommands


Manifest 1.24.904.1 Microsoft365DSC {Assert-M365DSCBlueprint, Compare-M365DSCConfigurations, Confirm-M365DSCDependencies, Export-M365DSCConfig... Manifest 1.24.724.1 Microsoft365DSC {Assert-M365DSCBlueprint, Compare-M365DSCConfigurations, Confirm-M365DSCDependencies, Export-M365DSCConfig... Manifest 1.24.717.1 Microsoft365DSC {Assert-M365DSCBlueprint, Compare-M365DSCConfigurations, Confirm-M365DSCDependencies, Export-M365DSCConfig...

warrenstark commented 2 weeks ago

Other output is a lot longer - just grabbing it

warrenstark commented 2 weeks ago

Transcript has the best version of the other data @FabienTschanz PowerShell_transcript.DESKTOP-O5FHP3R.Y6bE7Jvf.20240916155835.txt

FabienTschanz commented 2 weeks ago

PS C:\Windows\system32> Get-Module Microsoft365DSC -ListAvailable

Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version Name ExportedCommands

Manifest 1.24.904.1 Microsoft365DSC {Assert-M365DSCBlueprint, Compare-M365DSCConfigurations, Confirm-M365DSCDependencies, Export-M365DSCConfig... Manifest 1.24.724.1 Microsoft365DSC {Assert-M365DSCBlueprint, Compare-M365DSCConfigurations, Confirm-M365DSCDependencies, Export-M365DSCConfig... Manifest 1.24.717.1 Microsoft365DSC {Assert-M365DSCBlueprint, Compare-M365DSCConfigurations, Confirm-M365DSCDependencies, Export-M365DSCConfig...

Could you format the output using three backticks on the first line, then the output, and on a new last line again three backticks? This formats it as a code block with multiple lines. Thanks.

Edit: Nevermind that comment, saw it in the transcript. Thank you very much 👍

FabienTschanz commented 2 weeks ago

@warrenstark I saw that you have multiple Microsoft365DSC modules installed. Please run Uninstall-M365DSCOutdatedDependencies to clean them up, restart the PowerShell session, and then run the following new version. Test-ConvertToDSCObjectV5.ps1.txt

Edit: The parser isn't working correctly for you because it does not properly detect and import the DSC resources from the Microsoft365DSC module. I hope that we can find the reason why in the $ParseErrors variable after running the script parsing. You can execute that by yourself too and have a look at what's inside it (if there is any output by the script). I hope we can get to the bottom of that like this.

When I run the script with multiple modules installed, it still works, but I receive a couple of errors (one stating the the module is available multiple times, and two errors that the resource types for SPOTenantSettings and SPOHomeSite are not found). image

Edit2: What's the output of:

Edit3: Do you happen to have a separate computer where you could install the module and run the script as well? So that we have a double check and can make sure that it works "in general".

warrenstark commented 2 weeks ago

@FabienTschanz sorry for the couple days delay in response here, we had power work that knocked me off the air for a day, then a day catching up after.

It looks like clearing the old modules has worked, and creating a report from config seems to have worked out too. Attached the transcript and the output html file

Will be doing a bit more testing soon with bigger config files. PowerShell_transcript.DESKTOP-O5FHP3R.BYI3BTqZ.20240919081539.txt test.html.txt

FabienTschanz commented 2 weeks ago

@warrenstark No problem, we all have our hands busy with work. Sweet that it's working now, the output doesn't indicate any issues. Thank you for testing with some bigger config files, interested in those results. Thank you 👍

warrenstark commented 1 week ago

Testing with bigger configs working well - I think for my case here we can consider the issue resolved!

Thank you very much for your time Fabien

FabienTschanz commented 1 day ago

@ChrisFisherAllianceIT @AlexDiNicola @Numb3rten can you please try again with the latest version 1.24.1002.1? There are some fixes in it.