microsoftgraph / msgraph-sdk-powershell

Powershell SDK for Microsoft Graph
https://www.powershellgallery.com/packages/Microsoft.Graph
Other
715 stars 172 forks source link

`Microsoft.Graph.Identity.Governance` uses too much RAM to even be imported to Azure Automation Account #2981

Open o-l-a-v opened 1 month ago

o-l-a-v commented 1 month ago

Describe the bug

Microsoft.Graph.Identity.Governance requires so much RAM that it cannot be used in Azure Automation Account.

Simply importing Microsoft.Graph.Identity.Governance v2.23.0 gobbles up more than 500MB of RAM / memory!

This is especially a problem when running in memory restricted environments like Azure Automation Account.

Testing with v2.23.0

Screenshots going from pwsh -noprofile with PowerShell v7.4.5 x64 on Windows 11 23H2.

Clean start, idles at around 30 MB.

image

Import Microsoft.Graph.Authentication, does not take up much. Great.

image

Importing Microsoft.Graph.Identity.Governance makes pwsh.exe idle at almost 500 MB. Ouch. I've seen this idle around 500-600 MB at other times.

image

cookie-monster

Expected behavior

Don't take hundreds of MB of RAM.

How to reproduce

See "Describe the bug"

SDK Version

2.23.0

Latest version known to work for scenario above?

No response

Known Workarounds

In resource limited environments, like Azure Automation Account, use Microsoft.Graph.Authentication only with cmdlet Invoke-MgGraphRequest to do API requests, instead of using other resource heavy Microsoft.Graph modules.

In my example, replace:

Microsoft.Graph.Identity.Governance\Get-MgIdentityGovernancePrivilegedAccessGroupAssignmentScheduleInstance -All -Filter (
    'groupId eq ''{0}'' and endDateTime ne null' -f $PimGroup.'id'
)

with

(
    Microsoft.Graph.Authentication\Invoke-MgGraphRequest -Method 'Get' -Uri (
        [uri]::EscapeUriString(
            'https://graph.microsoft.com/v1.0/identityGovernance/privilegedAccess/group/assignmentScheduleInstances?' +
            ('$filter=groupId eq ''{0}'' and endDateTime ne null' -f $PimGroup.'id')
        )
    )
).'value'

You'll have to handle paging yourself with this approach.

Other

Maybe it'd be better to build and ship a single .psm1 file, instead of dot sourcing?

o-l-a-v commented 1 month ago

As v2.24.0 was just released and I'm still at work:

image

It seems to be even worse.

o-l-a-v commented 2 weeks ago

Simply importing Microsoft.Graph.Beta.Identity.Governance v2.24.0 crashes Automation Account with System.OutOfMemoryException. 😄

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Text.StringBuilder.ToString()
   at System.IO.StreamReader.ReadToEnd()
   at System.Management.Automation.ExternalScriptInfo.ReadScriptContents()
   at System.Management.Automation.ExternalScriptInfo.get_ScriptContents()
   at System.Management.Automation.ExternalScriptInfo.GetScriptBlockAst()
   at System.Management.Automation.ExternalScriptInfo.GetRequiresData()
   at System.Management.Automation.ExternalScriptInfo.get_RequiresElevation()
   at System.Management.Automation.CommandDiscovery.VerifyElevatedPrivileges(ExternalScriptInfo scriptInfo)
   at System.Management.Automation.CommandDiscovery.VerifyScriptRequirements(ExternalScriptInfo scriptInfo, ExecutionContext context)
   at System.Management.Automation.CommandDiscovery.CreateScriptProcessorForSingleShell(ExternalScriptInfo scriptInfo, ExecutionContext context, Boolean useLocalScope, SessionStateInternal sessionState)
   at System.Management.Automation.CommandDiscovery.LookupCommandProcessor(CommandInfo commandInfo, CommandOrigin commandOrigin, Nullable`1 useLocalScope, SessionStateInternal sessionState)
   at System.Management.Automation.CommandDiscovery.LookupCommandProcessor(String commandName, CommandOrigin commandOrigin, Nullable`1 useLocalScope)
   at System.Management.Automation.ExecutionContext.CreateCommand(String command, Boolean dotSource)
   at System.Management.Automation.PipelineOps.AddCommand(PipelineProcessor pipe, CommandParameterInternal[] commandElements, CommandBaseAst commandBaseAst, CommandRedirection[] redirections, ExecutionContext context)
   at System.Management.Automation.PipelineOps.InvokePipeline(Object input, Boolean ignoreInput, CommandParameterInternal[][] pipeElements, CommandBaseAst[] pipeElementAsts, CommandRedirection[][] commandRedirections, FunctionContext funcContext)
   at System.Management.Automation.Interpreter.ActionCallInstruction`6.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
$_.InvocationInfo.ScriptLineNumber = "3"
$_.InvocationInfo.InvocationName = "ForEach-Object"
$_.InvocationInfo.Line = "Get-ChildItem -Path 'C:\usr\src\PSModules\Microsoft.Graph.Beta.Identity.Governance\internal\..\exports' -Recurse -Include '*.ps1' -File | ForEach-Object { . $_.FullName }

Testing it locally ( Import-Module -Name 'Microsoft.Graph.Beta.Identity.Governance' ) the pwsh.exe process peaked at using around 1300 MB RAM while importing, while consuming around 900 MB ideling afterwards.