microsoft / ARI

Azure Resource Inventory - It's a Powerful tool to create EXCEL inventory from Azure Resources with low effort
MIT License
1.05k stars 378 forks source link

ResponsePayloadTooLarge #199

Closed maheshau1 closed 4 days ago

maheshau1 commented 1 month ago

Using 3.1.27.

Tried with and without -SkipPolicy -SkipAdvisory -SkipDiagram parameters.

Also tried -SkipDiagram -HeavyLoad

I am seeing the following with or without above parameteters.

Out of 10 subscriptions, the script worked fine on 9 of them. I'm only having an issue with 1 subscription or when I try to run it on the entire tenant.

DEBUG: 2024-07-03_17_43_26 - Total Number of Resources Found: 3957
az : ERROR: {
At C:\Users\winvmadmin\Downloads\ARI-main\ARI-main\AzureResourceInventory.ps1:597 char:49
+ ... ryResult = (az graph query -q $GraphQuery --subscriptions $FSubscri - ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ERROR: {:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

    "code": "BadRequest",
    "message": "Please provide below info when asking for support: timestamp = 2024-07-03T07:44:08.8214276Z, correlationId = fa7645b7-c780-42ca-96ad-4625e9879662.",
    "details": [
        {
            "code": "ResponsePayloadTooLarge",
            "message": "Response payload size is 94330584, and has exceeded the limit of 16777216. Please consider querying less data at a time and make paginated call if needed."
        }
    ]
}
You cannot call a method on a null-valued expression.
At C:\Users\winvmadmin\Downloads\ARI-main\ARI-main\AzureResourceInventory.ps1:597 char:33
+ ...             $QueryResult = (az graph query -q $GraphQuery --subscript ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

az : ERROR: {
At C:\Users\winvmadmin\Downloads\ARI-main\ARI-main\AzureResourceInventory.ps1:597 char:49
+ ... ryResult = (az graph query -q $GraphQuery --subscriptions $FSubscri - ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ERROR: {:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

    "code": "BadRequest",
    "message": "Please provide below info when asking for support: timestamp = 2024-07-03T07:44:15.8046345Z, correlationId = 3fb4df28-0bd4-45e3-b958-0345cb4a6134.",
    "details": [
        {
            "code": "ResponsePayloadTooLarge",
            "message": "Response payload size is 19377946, and has exceeded the limit of 16777216. Please consider querying less data at a time and make paginated call if needed."
        }
    ]
}
You cannot call a method on a null-valued expression.
At C:\Users\winvmadmin\Downloads\ARI-main\ARI-main\AzureResourceInventory.ps1:597 char:33
+ ...             $QueryResult = (az graph query -q $GraphQuery --subscript ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

DEBUG: 2024-07-03_17_44_42 - Invoking Inventory Loop for Network Resources 
.
.
.
.
 DEBUG: 2024-07-03_17_47_56 - Jobs Still Running: 9 

Running resources | where strlen(properties) > 123000 | summarize count() by ['type'],subscriptionId shows the following list: <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

TYPE | SUBSCRIPTIONID | COUNT_ -- | -- | -- microsoft.insights/workbooktemplates | * | 2 microsoft.network/applicationgateways | * | 2 microsoft.network/networksecuritygroups | * | 1 microsoft.resources/templatespecs/versions | * | 382 microsoft.logic/workflows | * | 9

Tried the following:

Claudio-Merola commented 1 month ago

Hello there @maheshau1,

I have made some changes to the script to try to address these issues and prevent leaving resources out of the script, can you try the version 3.1.28 and see how it goes?

Thanks

maheshau1 commented 1 month ago

Thank you for the prompt response @Claudio-Merola. I ran the updated .ps1 script, and while it exported the results, the data is not accurate. Tried with -SkipDiagram -HeavyLoad -debug parameters too and results were the same.

In the Excel file, I only see the Overview, Advisory, Policy, SecurityCenter, and Subscriptions tabs. In the Overview tab, the script has only exported SecurityCenter, which has a size of 0.

Here is the output from debug.

PS C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main> ./AzureResourceInventory.ps1 -TenantID **** -SubscriptionID **** -SecurityCenter -Diagram -Debug
DEBUG: 2024-07-04_08_44_34 - Debbuging Mode: On. ErrorActionPreference was set to "Continue", every error will be presented.
DEBUG: 2024-07-04_08_44_34 - Cleaning default variables
DEBUG: 2024-07-04_08_44_34 - Checking if -Online parameter will have to be forced.
DEBUG: 2024-07-04_08_44_34 - Starting Extractor function
DEBUG: 2024-07-04_08_44_34 - Starting checkAzCli function
Validating Az Cli..
DEBUG: 2024-07-04_08_44_36 - Current az cli version: azure-cli                         2.61.0
Validating Az Cli Extension..
DEBUG: 2024-07-04_08_44_37 - Current Resource-Graph Extension Version: 2.1.0
Validating ImportExcel Module..
DEBUG: 2024-07-04_08_44_40 - ImportExcel Module Version: 7.8.9
DEBUG: 2024-07-04_08_44_40 - Starting checkPS function
PowerShell Desktop Identified.

DEBUG: 2024-07-04_08_44_40 - Starting LoginSession function
Azure Cloud Environment: AzureCloud
DEBUG: 2024-07-04_08_44_47 - Tags will be ignored
DEBUG: 2024-07-04_08_44_47 - Number of Subscriptions Found: 
DEBUG: 2024-07-04_08_44_47 - Checking report folder: C:\AzureResourceInventory\
DEBUG: 2024-07-04_08_44_47 - Invoking Inventory Loop for Resources
ConvertFrom-Json : Cannot process argument because the value of argument "name" is not valid. Change the value of the "name" argument and run the operation again.
At C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\AzureResourceInventory.ps1:572 char:156
+ ... t 1000 --output json --only-show-errors).tolower() | ConvertFrom-Json
+                                                          ~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [ConvertFrom-Json], PSArgumentException
    + FullyQualifiedErrorId : Argument,Microsoft.PowerShell.Commands.ConvertFromJsonCommand

DEBUG: 2024-07-04_08_44_59 - Invoking Inventory Loop for Network Resources
DEBUG: 2024-07-04_08_45_01 - Invoking Inventory Loop for Backup Resources
DEBUG: 2024-07-04_08_45_02 - Invoking Inventory Loop for AVD Resources
DEBUG: 2024-07-04_08_45_04 - Invoking Inventory Loop for Resource Containers
DEBUG: 2024-07-04_08_45_06 - Invoking Inventory Loop for Policies Resources
DEBUG: 2024-07-04_08_45_08 - Invoking Inventory Loop for Advisories
DEBUG: 2024-07-04_08_45_11 - Invoking Inventory Loop for Security Resources
DEBUG: 2024-07-04_08_45_16 - Excel file:C:\AzureResourceInventory\AzureResourceInventory_Report_2024-07-04_08_45.xlsx
DEBUG: 2024-07-04_08_45_16 - Excel Table Style used: Light19
DEBUG: 2024-07-04_08_45_16 - Importing List of Unsupported Versions.
DEBUG: 2024-07-04_08_45_16 - Looking for the following file: C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Extras\Support.json
DEBUG: 2024-07-04_08_45_16 - Checking if Draw.io Diagram Job Should be Run.
DEBUG: 2024-07-04_08_45_16 - Starting Draw.io Diagram Processing Job.
DEBUG: 2024-07-04_08_45_16 - Checking If Should Run Security Center Job.
DEBUG: 2024-07-04_08_45_16 - Starting Security Job.
DEBUG: 2024-07-04_08_45_17 - Checking If Should Run Policy Job.
DEBUG: 2024-07-04_08_45_17 - Starting Policy Processing Job.
DEBUG: 2024-07-04_08_45_17 - Checking If Should Run Advisory Job.
DEBUG: 2024-07-04_08_45_17 - Starting Advisory Processing Job.
DEBUG: 2024-07-04_08_45_17 - Starting Subscriptions job.
DEBUG: 2024-07-04_08_45_18 - Starting Processing Jobs in Small Mode.
DEBUG: 2024-07-04_08_45_18 - Total Resources Being Processed: 30
DEBUG: 2024-07-04_08_45_18 - Resources Being Processed in ResourceJob_1: 28
DEBUG: 2024-07-04_08_45_19 - Starting Jobs Collector.
DEBUG: 2024-07-04_08_45_19 - Jobs Still Running: 1
DEBUG: 2024-07-04_08_45_22 - Jobs Still Running: 1
.
.
.

.
DEBUG: 2024-07-04_08_47_05 - Jobs Still Running: 1
DEBUG: 2024-07-04_08_47_08 - Jobs Still Running: 1
DEBUG: 2024-07-04_08_47_11 - Jobs Still Running: 1
DEBUG: 2024-07-04_08_47_14 - Jobs Compleated.
A variable with name 'ModRunA' already exists.
    + CategoryInfo          : ResourceExists: (ModRunA:String) [New-Variable], SessionStateException
    + FullyQualifiedErrorId : VariableAlreadyExists,Microsoft.PowerShell.Commands.NewVariableCommand
    + PSComputerName        : localhost

A variable with name 'ModJobA' already exists.
    + CategoryInfo          : ResourceExists: (ModJobA:String) [New-Variable], SessionStateException
    + FullyQualifiedErrorId : VariableAlreadyExists,Microsoft.PowerShell.Commands.NewVariableCommand
    + PSComputerName        : localhost

A variable with name 'ModValueA' already exists.
    + CategoryInfo          : ResourceExists: (ModValueA:String) [New-Variable], SessionStateException
    + FullyQualifiedErrorId : VariableAlreadyExists,Microsoft.PowerShell.Commands.NewVariableCommand
    + PSComputerName        : localhost

You cannot call a method on a null-valued expression.
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull
    + PSComputerName        : localhost

DEBUG: 2024-07-04_08_47_14 - Job ResourceJob_1 Returned: 24 Resource Types.
DEBUG: 2024-07-04_08_47_14 - Starting Reporting Phase.
DEBUG: 2024-07-04_08_47_14 - Running Offline, Gathering List Of Modules.
DEBUG: 2024-07-04_08_47_14 - Modules Found: 78
DEBUG: 2024-07-04_08_47_14 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\AzureAI.ps1'
DEBUG: 2024-07-04_08_47_14 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\ComputerVision.ps1'
DEBUG: 2024-07-04_08_47_14 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\ContentMod.ps1'
DEBUG: 2024-07-04_08_47_15 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\ContentSafety.ps1'
DEBUG: 2024-07-04_08_47_15 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\CustomVision.ps1'
DEBUG: 2024-07-04_08_47_15 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\Databricks.ps1'
DEBUG: 2024-07-04_08_47_15 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\DocIntelligence.ps1'
DEBUG: 2024-07-04_08_47_15 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\EvtHub.ps1'
DEBUG: 2024-07-04_08_47_15 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\FaceAPI.ps1'
DEBUG: 2024-07-04_08_47_16 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\HealthInsights.ps1'
DEBUG: 2024-07-04_08_47_16 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\ImmersiveReader.ps1'
DEBUG: 2024-07-04_08_47_16 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\Language.ps1'
DEBUG: 2024-07-04_08_47_16 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\MachineLearning.ps1'
DEBUG: 2024-07-04_08_47_16 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\SearchServices.ps1'
DEBUG: 2024-07-04_08_47_17 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\SpeechService.ps1'
DEBUG: 2024-07-04_08_47_17 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\Streamanalytics.ps1'
DEBUG: 2024-07-04_08_47_17 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\Synapse.ps1'
DEBUG: 2024-07-04_08_47_17 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\Translator.ps1'
DEBUG: 2024-07-04_08_47_17 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Analytics\WrkSpace.ps1'
DEBUG: 2024-07-04_08_47_17 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\APPSERVICEPLAN.ps1'
DEBUG: 2024-07-04_08_47_18 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\APPSERVICES.ps1'
DEBUG: 2024-07-04_08_47_18 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\ARCServers.ps1'
DEBUG: 2024-07-04_08_47_18 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\ARO.ps1'
DEBUG: 2024-07-04_08_47_18 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\AVD.ps1'
DEBUG: 2024-07-04_08_47_18 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\CloudServices.ps1'
DEBUG: 2024-07-04_08_47_19 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\VM.ps1'
DEBUG: 2024-07-04_08_47_19 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Compute\VMWare.ps1'
DEBUG: 2024-07-04_08_47_19 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Containers\AKS.ps1'
DEBUG: 2024-07-04_08_47_19 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Containers\CONTAINER.ps1'
DEBUG: 2024-07-04_08_47_19 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Containers\REGISTRIES.ps1'
DEBUG: 2024-07-04_08_47_19 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Containers\VMSS.ps1'
DEBUG: 2024-07-04_08_47_20 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\CosmosDB.ps1'
DEBUG: 2024-07-04_08_47_20 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\MariaDB.ps1'
DEBUG: 2024-07-04_08_47_20 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\MySQL.ps1'
DEBUG: 2024-07-04_08_47_20 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\MySQLflexible.ps1'
DEBUG: 2024-07-04_08_47_20 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\POSTGRE.ps1'
DEBUG: 2024-07-04_08_47_20 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\POSTGREFlexible.ps1'
DEBUG: 2024-07-04_08_47_21 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\Purview.ps1'
DEBUG: 2024-07-04_08_47_21 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\RedisCache.ps1'
DEBUG: 2024-07-04_08_47_21 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\SQLDB.ps1'
DEBUG: 2024-07-04_08_47_21 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\SQLMI.ps1'
DEBUG: 2024-07-04_08_47_21 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\SQLMIDB.ps1'
DEBUG: 2024-07-04_08_47_22 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\SQLPOOL.ps1'
DEBUG: 2024-07-04_08_47_22 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\SQLSERVER.ps1'
DEBUG: 2024-07-04_08_47_22 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Data\SQLVM.ps1'
DEBUG: 2024-07-04_08_47_22 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Infrastructure\AppGW.ps1'
DEBUG: 2024-07-04_08_47_22 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Infrastructure\AutomationAcc.ps1'
DEBUG: 2024-07-04_08_47_22 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Infrastructure\AvSet.ps1'
DEBUG: 2024-07-04_08_47_23 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Infrastructure\BASTION.ps1'
DEBUG: 2024-07-04_08_47_23 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Infrastructure\Frontdoor.ps1'
DEBUG: 2024-07-04_08_47_23 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Infrastructure\RecoveryVault.ps1'
DEBUG: 2024-07-04_08_47_23 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Infrastructure\Vault.ps1'
DEBUG: 2024-07-04_08_47_23 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Integration\APIM.ps1'
DEBUG: 2024-07-04_08_47_24 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Integration\AppInsights.ps1'
DEBUG: 2024-07-04_08_47_24 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Integration\Backup.ps1'
DEBUG: 2024-07-04_08_47_24 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Integration\IOTHubs.ps1'
DEBUG: 2024-07-04_08_47_24 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Integration\ServiceBUS.ps1'
DEBUG: 2024-07-04_08_47_24 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\AzureFirewall.ps1'
DEBUG: 2024-07-04_08_47_24 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\Connections.ps1'
DEBUG: 2024-07-04_08_47_25 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\ExpressRoute.ps1'
DEBUG: 2024-07-04_08_47_25 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\LoadBalancer.ps1'
DEBUG: 2024-07-04_08_47_25 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\NATGAteway.ps1'
DEBUG: 2024-07-04_08_47_25 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\NetworkInterface.ps1'
DEBUG: 2024-07-04_08_47_25 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\NetworkSecurityGroup.ps1'
DEBUG: 2024-07-04_08_47_26 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\PrivateDNS.ps1'
DEBUG: 2024-07-04_08_47_26 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\PrivateEndpoint.ps1'
DEBUG: 2024-07-04_08_47_26 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\PublicDNS.ps1'
DEBUG: 2024-07-04_08_47_26 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\PublicIP.ps1'
DEBUG: 2024-07-04_08_47_26 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\ROUTETABLE.ps1'
DEBUG: 2024-07-04_08_47_26 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\TrafficManager.ps1'
DEBUG: 2024-07-04_08_47_27 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\VirtualNetwork.ps1'
DEBUG: 2024-07-04_08_47_27 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\VirtualWAN.ps1'
DEBUG: 2024-07-04_08_47_27 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\VNETGTW.ps1'
DEBUG: 2024-07-04_08_47_27 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Networking\vNETPeering.ps1'
DEBUG: 2024-07-04_08_47_27 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Storage\DataExplorerCluster.ps1'
DEBUG: 2024-07-04_08_47_27 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Storage\NetApp.ps1'
DEBUG: 2024-07-04_08_47_28 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Storage\StorageAcc.ps1'
DEBUG: 2024-07-04_08_47_28 - Running Module: 'C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Modules\Storage\VMDisk.ps1'
DEBUG: 2024-07-04_08_47_28 - Resource Reporting Phase Done.
DEBUG: 2024-07-04_08_47_28 - Checking if Should Generate Security Center Sheet.
DEBUG: 2024-07-04_08_47_28 - Generating Security Center Sheet.
DEBUG: 2024-07-04_08_47_28 - Looking for the following file: C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Extras\SecurityCenter.ps1
DEBUG: 2024-07-04_08_47_31 - Checking if Should Generate Policy Sheet.
DEBUG: 2024-07-04_08_47_31 - Generating Policy Sheet.
DEBUG: 2024-07-04_08_47_31 - Looking for the following file: C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Extras\Policy.ps1
DEBUG: 2024-07-04_08_47_31 - Checking if Should Generate Advisory Sheet.
DEBUG: 2024-07-04_08_47_31 - Generating Advisor Sheet.
DEBUG: 2024-07-04_08_47_31 - Looking for the following file: C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Extras\Advisory.ps1
DEBUG: 2024-07-04_08_47_32 - Generating Subscription sheet for:  Subscriptions.
DEBUG: 2024-07-04_08_47_32 - Looking for the following file: C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Extras\Subscriptions.ps1
DEBUG: 2024-07-04_08_47_33 - Generating Overview sheet (Charts).
DEBUG: 2024-07-04_08_47_33 - Looking for the following file: C:\Users\winvmadmin\Downloads\ARI-main-updated\ARI-main\Extras\Charts.ps1
DEBUG: 2024-07-04_08_47_35 - Finished Charts Phase.
Report Complete. Total Runtime was: 3.02 Minutes
Total Resources: 30
Total Advisories: 1000
Total Policies: 12
Total Security Advisories: 1000

Excel file saved at: C:\AzureResourceInventory\AzureResourceInventory_Report_2024-07-04_08_45.xlsx

Draw.io Diagram file saved at: C:\AzureResourceInventory\AzureResourceInventory_Diagram_2024-07-04_08_45.xml 
Claudio-Merola commented 1 month ago

Hi @maheshau1

Are you running the script in Powershell Core or Windows Powershell?

maheshau1 commented 1 month ago

Hello @Claudio-Merola

Using a Desktop version of PowerShell.

PS C:\Users\winvmadmin> $PSVersionTable

Name                           Value                                                                                                                                                                                                   
----                           -----                                                                                                                                                                                                   
PSVersion                      5.1.22621.3672                                                                                                                                                                                          
PSEdition                      Desktop                                                                                                                                                                                                 
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                                                                 
BuildVersion                   10.0.22621.3672                                                                                                                                                                                         
CLRVersion                     4.0.30319.42000                                                                                                                                                                                         
WSManStackVersion              3.0                                                                                                                                                                                                     
PSRemotingProtocolVersion      2.3                                                                                                                                                                                                     
SerializationVersion           1.1.0.1                                                                                                                                                                                                

I noticed your commit to enforce Core and I installed PowerShell Core.


Name                           Value
----                           -----
PSVersion                      7.4.3
PSEdition                      Core
GitCommitId                    7.4.3
OS                             Microsoft Windows 10.0.22621
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

I am still seeing similar results as above when running script from PowerShell Core. Script finishes but only extracts 30 resources.

DEBUG: 2024-07-04_23_48_56 - Debbuging Mode: On. ErrorActionPreference was set to "Continue", every error will be presented.
DEBUG: 2024-07-04_23_48_56 - Cleaning default variables
DEBUG: 2024-07-04_23_48_56 - Checking if -Online parameter will have to be forced.
DEBUG: 2024-07-04_23_48_56 - Starting Extractor function
DEBUG: 2024-07-04_23_48_56 - **Powershell Edition: Core**
DEBUG: 2024-07-04_23_48_56 - **Powershell Version: 7.4.3**
DEBUG: 2024-07-04_23_48_56 - Starting checkAzCli function
Validating Az Cli..
DEBUG: 2024-07-04_23_48_58 - Current az cli version: azure-cli                         2.61.0
Validating Az Cli Extension..
DEBUG: 2024-07-04_23_48_59 - Current Resource-Graph Extension Version: 2.1.0
Validating ImportExcel Module..
DEBUG: 2024-07-04_23_49_04 - ImportExcel Module Version: ..
Trying to install ImportExcel Module..
DEBUG: 2024-07-04_23_49_13 - Starting checkPS function
PowerShell Desktop Identified.

DEBUG: 2024-07-04_23_49_13 - Starting LoginSession function
Azure Cloud Environment: AzureCloud
DEBUG: 2024-07-04_23_49_21 - Tags will be ignored
DEBUG: 2024-07-04_23_49_21 - Number of Subscriptions Found: 1
DEBUG: 2024-07-04_23_49_21 - Checking report folder: C:\AzureResourceInventory\
DEBUG: 2024-07-04_23_49_21 - Invoking Inventory Loop for Resources

First error:

ConvertFrom-Json: C:\Users\winvmadmin\Documents\dev\ARI\AzureResourceInventory.ps1:575
Line |
 575 |  … t 1000 --output json --only-show-errors).tolower() | ConvertFrom-Json
     |                                                         ~~~~~~~~~~~~~~~~
     | The provided JSON includes a property whose name is an empty string, this is only supported using the
     | -AsHashTable switch.

Script continues

DEBUG: 2024-07-04_23_49_31 - Invoking Inventory Loop for Network Resources
.
.
.
.
.
.
DEBUG: 2024-07-04_23_51_59 - Looking for the following file: C:\Users\winvmadmin\Documents\dev\ARI\Extras\Charts.ps1
DEBUG: 2024-07-04_23_52_01 - Finished Charts Phase.
Report Complete. Total Runtime was: 3.09 Minutes
Total Resources: 30
Total Advisories: 1000
Total Policies: 12
Total Security Advisories: 1000

Excel file saved at: C:\AzureResourceInventory\AzureResourceInventory_Report_2024-07-04_23_49.xlsx

Draw.io Diagram file saved at: C:\AzureResourceInventory\AzureResourceInventory_Diagram_2024-07-04_23_49.xml
Claudio-Merola commented 1 month ago

Hi @maheshau1

Yea, I was targeting the wrong issue...

Apparently the issue is related with WAM, so the authentication itself was failing. I have changed the script to basically disable WAM.

I don't really like that type of approach, but it will work until a fix is provided by the az team.

Can you try the version 3.1.29 and see how it goes?

maheshau1 commented 1 month ago

Same results using 3.1.29

Script starts running

DEBUG: 2024-07-05_00_32_38 - Debbuging Mode: On. ErrorActionPreference was set to "Continue", every error will be presented.
DEBUG: 2024-07-05_00_32_38 - Cleaning default variables
DEBUG: 2024-07-05_00_32_38 - Checking if -Online parameter will have to be forced.
DEBUG: 2024-07-05_00_32_38 - Starting Extractor function
DEBUG: 2024-07-05_00_32_38 - Powershell Edition: Core
DEBUG: 2024-07-05_00_32_38 - Powershell Version: 7.4.3
DEBUG: 2024-07-05_00_32_38 - Starting checkAzCli function
Validating Az Cli..
DEBUG: 2024-07-05_00_32_41 - Current az cli version: azure-cli                         2.61.0
Validating Az Cli Extension..
DEBUG: 2024-07-05_00_32_42 - Current Resource-Graph Extension Version: 2.1.0
Validating ImportExcel Module..
DEBUG: 2024-07-05_00_32_49 - ImportExcel Module Version: ..
DEBUG: 2024-07-05_00_32_49 - Starting checkPS function
PowerShell Desktop Identified.

DEBUG: 2024-07-05_00_32_49 - Starting LoginSession function
Azure Cloud Environment: AzureCloud
DEBUG: 2024-07-05_00_32_57 - Tags will be ignored
DEBUG: 2024-07-05_00_32_57 - Number of Subscriptions Found: 1
DEBUG: 2024-07-05_00_32_57 - Checking report folder: C:\AzureResourceInventory\
DEBUG: 2024-07-05_00_32_57 - Invoking Inventory Loop for Resources

First error

ConvertFrom-Json: C:\Users\winvmadmin\Documents\dev\ARI\AzureResourceInventory.ps1:577
Line |
 577 |  … t 1000 --output json --only-show-errors).tolower() | ConvertFrom-Json
     |                                                         ~~~~~~~~~~~~~~~~
     | The provided JSON includes a property whose name is an empty string, this is only supported using the -AsHashTable switch.

Script continues and output includes only 30 resources

DEBUG: 2024-07-05_00_33_08 - Invoking Inventory Loop for Network Resources
.
.
.
DEBUG: 2024-07-05_00_33_25 - Total Resources Being Processed: 30
.
.
DEBUG: 2024-07-05_00_35_39 - Finished Charts Phase.
Report Complete. Total Runtime was: 3.02 Minutes
Total Resources: 30
Total Advisories: 1000
Total Policies: 12
Total Security Advisories: 1000

Excel file saved at: C:\AzureResourceInventory\AzureResourceInventory_Report_2024-07-05_00_33.xlsx

Draw.io Diagram file saved at: C:\AzureResourceInventory\AzureResourceInventory_Diagram_2024-07-05_00_33.xml

⚠️ To test further, I used different subscription from tenant and script worked fine. Out of 10 subscriptions, the script worked fine on 9 of them even yesterday (v 3.1.27). I'm only having an issue with 1 subscription or when I try to run it on the entire tenant.

Claudio-Merola commented 1 month ago

Hi @maheshau1

Apparently there is a resource in that environment with a blank name. That why the script is failing in that part.

This is very odd and I don't think is normal for any resource type to don't have a name.

Can you find that resource in Resource Graph Explorer so we can try to understand what is going on?

maheshau1 commented 1 month ago

Used below queries but not seeing anything in output:

resources
| where name == ''
| project id, type, location, resourceGroup
resources
| where strlen(name) <= 1
| project id, type, location, resourceGroup, name
Claudio-Merola commented 1 month ago

Hello @maheshau1

The "name" can be basically any of the attribute's name of the resource...

I made a workaround specific for you, but it might help other people with the same issue.

Can you run the latest version of the script (3.1.30) using PowerShell Core and see how it goes?

maheshau1 commented 1 month ago

Thank you, @Claudio-Merola. The script still shows an error (see the output below). Additionally, it did not list all the resources. In my subscription, I have more than 3000 resources, but the total resources exported are only 1029.

The Excel file was also corrupt, displaying an error message: "Excel was able to open the file by repairing or removing the unreadable content." In the Excel export, I can only see the Overview, Advisory, Policy, and Subscription tabs; no other tabs are present.

PS C:\Users\winvmadmin\Documents\dev\ARI> ./AzureResourceInventory.ps1 -TenantID *** -SubscriptionID *** -Debug
DEBUG: 2024-07-06_08_23_09 - Debbuging Mode: On. ErrorActionPreference was set to "Continue", every error will be presented.
DEBUG: 2024-07-06_08_23_09 - Cleaning default variables
DEBUG: 2024-07-06_08_23_09 - Checking if -Online parameter will have to be forced.
DEBUG: 2024-07-06_08_23_09 - Starting Extractor function
DEBUG: 2024-07-06_08_23_09 - Powershell Edition: Core
DEBUG: 2024-07-06_08_23_09 - Powershell Version: 7.4.3
DEBUG: 2024-07-06_08_23_09 - Starting checkAzCli function
Validating Az Cli..
DEBUG: 2024-07-06_08_23_11 - Current az cli version: azure-cli                         2.61.0
Validating Az Cli Extension..
DEBUG: 2024-07-06_08_23_12 - Current Resource-Graph Extension Version: 2.1.0
Validating ImportExcel Module..
DEBUG: 2024-07-06_08_23_13 - ImportExcel Module Version: ..
DEBUG: 2024-07-06_08_23_13 - Starting checkPS function
PowerShell Desktop Identified.

DEBUG: 2024-07-06_08_23_13 - Starting LoginSession function
Azure Cloud Environment: AzureCloud
DEBUG: 2024-07-06_08_23_22 - Tags will be ignored
DEBUG: 2024-07-06_08_23_22 - Number of Subscriptions Found: 1
DEBUG: 2024-07-06_08_23_22 - Checking report folder: C:\AzureResourceInventory\
DEBUG: 2024-07-06_08_23_22 - Invoking Inventory Loop for Resources
DEBUG: 2024-07-06_08_23_34 - Invoking Inventory Loop for Network Resources
DEBUG: 2024-07-06_08_23_36 - Invoking Inventory Loop for Backup Resources
DEBUG: 2024-07-06_08_23_38 - Invoking Inventory Loop for AVD Resources
DEBUG: 2024-07-06_08_23_40 - Invoking Inventory Loop for Resource Containers
DEBUG: 2024-07-06_08_23_41 - Invoking Inventory Loop for Policies Resources
DEBUG: 2024-07-06_08_23_43 - Invoking Inventory Loop for Advisories
DEBUG: 2024-07-06_08_23_46 - Excel file:C:\AzureResourceInventory\AzureResourceInventory_Report_2024-07-06_08_23.xlsx
DEBUG: 2024-07-06_08_23_46 - Excel Table Style used: Light19
DEBUG: 2024-07-06_08_23_46 - Importing List of Unsupported Versions.
DEBUG: 2024-07-06_08_23_46 - Looking for the following file: C:\Users\winvmadmin\Documents\dev\ARI\Extras\Support.json
DEBUG: 2024-07-06_08_23_46 - Checking if Draw.io Diagram Job Should be Run.
DEBUG: 2024-07-06_08_23_46 - Starting Draw.io Diagram Processing Job.
DEBUG: 2024-07-06_08_23_47 - Checking If Should Run Security Center Job.
DEBUG: 2024-07-06_08_23_47 - Checking If Should Run Policy Job.
DEBUG: 2024-07-06_08_23_47 - Starting Policy Processing Job.
DEBUG: 2024-07-06_08_23_47 - Checking If Should Run Advisory Job.
DEBUG: 2024-07-06_08_23_47 - Starting Advisory Processing Job.
DEBUG: 2024-07-06_08_23_47 - Starting Subscriptions job.
DEBUG: 2024-07-06_08_23_48 - Starting Processing Jobs in Medium Mode.
DEBUG: 2024-07-06_08_23_48 - Total Resources Being Processed: 1029
DEBUG: 2024-07-06_08_23_48 - Resources Being Processed in ResourceJob_1: 1028
WARNING: Resulting JSON is truncated as serialization has exceeded the set depth of 25.
DEBUG: 2024-07-06_08_23_49 - Starting Jobs Collector.
DEBUG: 2024-07-06_08_23_49 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_23_52 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_23_56 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_23_59 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_02 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_05 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_08 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_11 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_14 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_17 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_20 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_23 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_26 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_29 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_32 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_35 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_38 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_41 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_44 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_47 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_50 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_53 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_56 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_24_59 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_02 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_05 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_08 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_11 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_14 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_17 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_20 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_23 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_26 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_29 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_32 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_35 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_38 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_41 - Jobs Still Running: 1
DEBUG: 2024-07-06_08_25_44 - Jobs Compleated.
ConvertFrom-Json: The provided JSON includes a property whose name is an empty string, this is only supported using the -AsHashTable switch.
DEBUG: 2024-07-06_08_25_44 - Job ResourceJob_1 Returned: 0 Resource Types.
DEBUG: 2024-07-06_08_25_44 - Starting Reporting Phase.
DEBUG: 2024-07-06_08_25_44 - Running Offline, Gathering List Of Modules.
DEBUG: 2024-07-06_08_25_44 - Modules Found: 78
DEBUG: 2024-07-06_08_25_49 - Resource Reporting Phase Done.
DEBUG: 2024-07-06_08_25_49 - Checking if Should Generate Security Center Sheet.
DEBUG: 2024-07-06_08_25_49 - Checking if Should Generate Policy Sheet.
DEBUG: 2024-07-06_08_25_49 - Generating Policy Sheet.
DEBUG: 2024-07-06_08_25_49 - Looking for the following file: C:\Users\winvmadmin\Documents\dev\ARI\Extras\Policy.ps1
DEBUG: 2024-07-06_08_25_51 - Checking if Should Generate Advisory Sheet.
DEBUG: 2024-07-06_08_25_51 - Generating Advisor Sheet.
DEBUG: 2024-07-06_08_25_51 - Looking for the following file: C:\Users\winvmadmin\Documents\dev\ARI\Extras\Advisory.ps1
DEBUG: 2024-07-06_08_25_52 - Generating Subscription sheet for: 1 Subscriptions.
DEBUG: 2024-07-06_08_25_52 - Looking for the following file: C:\Users\winvmadmin\Documents\dev\ARI\Extras\Subscriptions.ps1
DEBUG: 2024-07-06_08_25_52 - Generating Overview sheet (Charts).
DEBUG: 2024-07-06_08_25_52 - Looking for the following file: C:\Users\winvmadmin\Documents\dev\ARI\Extras\Charts.ps1
DEBUG: 2024-07-06_08_25_55 - Finished Charts Phase.
Report Complete. Total Runtime was: 2.76 Minutes
Total Resources: 1029
Total Advisories: 727
Total Policies: 12

Excel file saved at: C:\AzureResourceInventory\AzureResourceInventory_Report_2024-07-06_08_23.xlsx

Draw.io Diagram file saved at: C:\AzureResourceInventory\AzureResourceInventory_Diagram_2024-07-06_08_23.xml

PS C:\Users\winvmadmin\Documents\dev\ARI>
Claudio-Merola commented 1 month ago

Hello @maheshau1

Regarding the total number of resources, never mind, I found I bug in the skip token usage by the script, should be fixed now

I assume the corrupt excel is due to the warning we are seeing in the log. I think I fixed that in the new version 3.1.32, can you try that version and see how it goes?

maheshau1 commented 1 month ago

Thanks, @Claudio-Merola. We are back to the ResponsePayloadTooLarge error. You might notice that in the first log I posted when I was using Version: 3.1.27, the script was able to identify 3957 resources.

I am now using Version: 3.1.35

PS C:\Users\winvmadmin\Documents\dev\ARI> ./AzureResourceInventory.ps1 -TenantID **** -SubscriptionID **** -Lite
Validating Az Cli..
Validating Az Cli Extension..
Validating ImportExcel Module..
PowerShell Desktop Identified.

Azure Cloud Environment: AzureCloud
ERROR: {ventory [3% Complete.                                                                                        ]
    "code": "BadRequest",
    "message": "Please provide below info when asking for support: timestamp = 2024-07-12T00:42:32.5674642Z, correlationId = b8d0d097-836c-4939-8fb1-f4914e5ce0f8.",
    "details": [
        {
            "code": "ResponsePayloadTooLarge",
            "message": "Response payload size is 27135752, and has exceeded the limit of 16777216. Please consider querying less data at a time and make paginated call if needed."
        }
    ]
Report Complete. Total Runtime was: 3.06 Minutes
Total Resources: 1029
Total Advisories: 2514
Total Policies: 12

Excel file saved at: C:\AzureResourceInventory\AzureResourceInventory_Report_2024-07-12_10_42.xlsx

Draw.io Diagram file saved at: C:\AzureResourceInventory\AzureResourceInventory_Diagram_2024-07-12_10_42.xml
code2exe commented 1 month ago

Dear @Claudio-Merola I also get this error when running the script on a large environment

image (3)

Claudio-Merola commented 1 month ago

Hello @code2exe and @maheshau1

That error happens because some resources have too many details, usually in the "properties" field. In the past we worked around that by only bringing the resources which properties were smaller than X size. But that would lead to misleading the correct number of resources in the inventory.

So we are trying to address that in a more "correct" way by automatically bringing smaller chunks of data.

I have updated the script to bring chunks of 500 resources instead of 1000 when that error occurs.

Can you guys try the version 3.1.36 and see if it works in those same environments?

maheshau1 commented 1 month ago

I just completed another test with version 3.1.37

Results were the same as my last comment: ResponsePayloadTooLarge error and I am not getting all the resources in a spreadsheet.

Claudio-Merola commented 1 month ago

Hi @maheshau1

I decreased the size of the chunks to 200. Can you test the version 3.1.38 and see if the error still occurs?

maheshau1 commented 1 month ago

@Claudio-Merola last commit that I can see is version 3.1.37. I am unable to see version 3.1.38.

Claudio-Merola commented 1 month ago

@maheshau1 , I'm sorry, my bad. Version 3.1.38 is out now.

maheshau1 commented 1 month ago

Still the same @Claudio-Merola

image

Given that this might be an isolated issue within my environment, I am comfortable closing this issue. Your assistance has been invaluable, and I appreciate all your support. Thank you.

github-actions[bot] commented 1 week ago

This Issue is stale and may be automatically close soon.