"Jobs Still Running: 1" gets stuck

[alecorgit]

Hello, we are having a stuck issue on 1 subscription, we have already used the AzureResourceInventory script on 11 other subscriptions that have completed the Excel reports.

Could you help us understand why we cannot complete this Job?

The command executed is: .\AzureResourceInventory.ps1 -TenantID yyyyyyyy -SubscriptionID aaaaaaaaaaa -ReportName AzResInv -debug

We also tried using -HeavyLoad, -Lite, -Online, -DiagramFullEnvironment, -ReportDir, -SkipDiagram, -SkipAdvisory, -SkipPolicy parameters, but we always gets stuck for hours (up to 8 hours till the script was manually terminated), the output is:

Azure Resource Inventory Reporting (55600) Resources
Processing Resource Jobs 96% Complete.

DEBUG: Starting Processing Jobs in Heavy Mode.
DEBUG: Starting Jobs Collector.
DEBUG: Jobs Still Running: 23
…
DEBUG: Jobs Still Running: 1
DEBUG: Jobs Still Running: 1
DEBUG: Jobs Still Running: 1
DEBUG: Jobs Still Running: 1
DEBUG: Jobs Still Running: 1 (still running)

I saw the issue #140 but our server has 8 vCPU (20% used while "Jobs Still Running: 1") and 64 GB RAM (23 GB RAM used while "jobs Still Running: 1")

Kind regards

[red-erik]

Hello, I have the same exact problem with a server (VM) with Standard_D14_v2 (16 Vcpu and 112 Gb RAM). For better troubleshooting, it would be nice to have jobs with a "specific and unique" name.

Regards, Red.

[Claudio-Merola]

Hello @alecorgit

Do you have many Azure Load Balancers in your environment?

I ask because we made the Load Balancer processing module some time ago and it has many inner loops, which might be causing these recurring issues.

If you are not running the script with -Online, Is possible for you to delete the file \modules\networking/loadbalancer.ps1 and run the script again to see if that behavior occur?

Then we can confirm this is issue.

Thanks

[alecorgit]

Hello @Claudio-Merola, we have almost 200 Load Balancers in this subscription, removing the networking\loadbalancer.ps1 file helped us to move forward and complete the Azure Resource Inventory Reporting step. However now the script seems to get stuck on "DEBUG: Finished Charts Phase", but at least we understood what got stuck and we have the Excel report.

Thank you so much for your support and all your effort in developing this amazing product.

[Claudio-Merola]

@alecorgit

Thanks for helping on the tests.

If you do not use -Lite parameter, the "Chart Phase" will use the Excel APIs from the Excel installed in the computer, since you mentioned you are running the script from inside an Azure VM I assume you don't have Excel installed there, right?

If that's the case this might me one of the reasons for the script to get stuck in that phase.

Can you try to run it using the -Lite parameter and see if it still get stuck in that phase?

Regarding the Load Balancer, this issue is a little more complex, I will begin to work on a new version for the Load Balancer module since now we are sure the current version is causing problems.

[alecorgit]

Hello @Claudio-Merola, the server is an Azure VM without Excel, I tried -Lite parameter:

Validating ImportExcel Module.. DEBUG: ImportExcel Module Version: 7.8.6

DEBUG: Excel Table Style used: Light19

But the script is still running at DEBUG: Finished Charts Phase. On the same server the other subscriptions completed with or without -Lite Parameter.

[Claudio-Merola]

Oh, ok...

The script is stuck with the diagram...

Guess if you just run the script with the -SkipDiagram parameter it will not get stuck there. But the diagram is not going to be created.

[alecorgit]

Hi, I downloaded 3.1.11 version, I saw that Load Balancer module is already disabled (LoadBalancer.ps2) and executed the inventory with -SkipDiagram -debug - HeavyLoad. I can confirm -SkipDiagram works 👍🏻 :

Report Complete. Total Runtime was: 39.92 Minutes

[Claudio-Merola]

Hi @alecorgit

Thanks. I'm still working with the Load Balancer modules so it is still disabled.