search

microsoft / ARI

Azure Resource Inventory - It's a Powerful tool to create EXCEL inventory from Azure Resources with low effort
MIT License
1.12k stars 397 forks source link

I am having 29667 resource and the powershell is getting stuck #140

Closed ashrafcitius closed 1 year ago

ashrafcitius commented 1 year ago

While executing the powershell script the script is getting stuck, i have around 29667 resources, i does generate an excel sheet but it has missed a lot of things even its showing error regarding Pivot tables also it does generate any xml diagram sheet, please suggest what can be done?

Claudio-Merola commented 1 year ago

Hello @ashrafcitius

Where are you running the script? CloudShell, Windows 11 client computer, Mac?

Can you try to run the script with -Debug and -Lite parameters to see how it goes?

I have seen some problems when you do have some different types of resources with too many details in the properties for example

ashrafcitius commented 1 year ago

I am running this in windows 11, and I did added the -Debug and -Lite parameter and then too it gets stuck at DEBUG: Jobs still Running:300 and then it just continues saying the same and gets stuck

ashrafcitius commented 1 year ago

It gets stuck at

DEBUG: Starting subscription job DEBUG: Starting processing job DEBUG: Starting jobs collector DEBUG: Jobs still running:30 DEBUG: Jobs still running:30

And then it just continues to show the same after sometime it just get stuck.

Claudio-Merola commented 1 year ago

I believe I know what might be happening, I think it is starting too much processes to handle the load...

I just uploaded a version 3.1.06, can you try it using the new parameter -Heavy?

ashrafcitius commented 1 year ago

Do you meant to say > .\AzureResourceInventory.ps1 -Heavy -Debug

because i can see under the script there is a parameter as $HeavyLoad so should i mention -HeavyLoad or -Heavy?

ashrafcitius commented 1 year ago

I tried to execute > .\AzureResourceInventory.ps1 -Heavy -Debug -Lite and now it is stuck at below.

DEBUG: Jobs still running: 6

just FYI it did hit the Heavy parameter that you have written.

Claudio-Merola commented 1 year ago

sorry, is -HeavyLoad

Can you download the version 3.1.07 and give it a try?

Thanks

ashrafcitius commented 1 year ago

As you said i download the 3.1.07 and executed it with -HeavyLoad and now i am stuck as you can see below:

Seems like its taking time to process the resources and that is where it just get stuck as it contains a lot of properties and Resources.

DEBUG: Excel file:C:\AzureResourceInventory\AzureResourceInventory_Report_2023-08-28_19_24.xlsx DEBUG: Excel Table Style used: Light19 DEBUG: Importing List of Unsupported Versions. DEBUG: Looking for the following file: https://raw.githubusercontent.com/microsoft/ARI/main/Extras/Support.json DEBUG: Checking if Draw.io Diagram Job Should be Run. DEBUG: Starting Draw.io Diagram Processing Job. DEBUG: Checking If Should Run Security Center Job. DEBUG: Checking If Should Run Policy Job. DEBUG: Starting Policy Processing Job. DEBUG: Checking If Should Run Advisory Job. DEBUG: Starting Advisory Processing Job. DEBUG: Starting Subscriptions job. DEBUG: Starting Processing Jobs in Heavy Mode. DEBUG: Starting Jobs Collector. DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7 DEBUG: Jobs Still Running: 7

Claudio-Merola commented 1 year ago

It should take some time to process. But it should not be stuck, can you leave it like that for a few hours and see if the number of jobs get lower?

ashrafcitius commented 1 year ago

As you said I left it as it is for hours but the job did not get reduced also my CPU utilisation was 100% because of which my system itself got stuck.

Claudio-Merola commented 1 year ago

The processing power of you computer is not being able to handle that load.

I tried to add some extra resources in the script to slow a little bit the processing speed. it might take more time, perhaps some hours to complete, but I hope it will eventually not get stuck at 100%.

Try to download the last version and run with -HeavyLoad.

But if that do not work I'm afraid there is not going to be much else to do from this side.

github-actions[bot] commented 1 year ago

This Issue is stale and may be automatically close soon.