JulianHayward / Azure-MG-Sub-Governance-Reporting

Azure Governance Visualizer aka AzGovViz is a PowerShell script that captures Azure Governance related information such as Azure Policy, RBAC (a lot more) by polling Azure ARM, Storage and Microsoft Graph APIs.
MIT License
814 stars 293 forks source link

Failing due to case sensitivity of key 'timegrain/timeGrain' #235

Closed ReneRebsdorf closed 1 month ago

ReneRebsdorf commented 1 month ago

AzGovViz version master:69e5b56

CodeRunPlatform Azure DevOps

Describe the bug [AzAPICall 1.2.1] 'Getting Policy definitions for Management Group: 'REDACTED' ('REDACTED')' uri='https://management.azure.com/providers/Microsoft.Management/managementgroups/REDACTED/providers/Microsoft.Authorization/policyDefinitions?api-version=2021-06-01&$filter=policyType eq 'Custom'' Command 'ConvertFrom-Json' failed: Cannot convert the JSON string because it contains keys with different casing. Please use the -AsHashTable switch instead. The key that was attempted to be added to the existing key 'timegrain' was 'timeGrain'. [AzAPICall 1.2.1] 'Getting Policy definitions for Management Group: 'REDACTED' ('REDACTED')' uri='https://management.azure.com/providers/Microsoft.Management/managementgroups/REDACTED/providers/Microsoft.Authorization/policyDefinitions?api-version=2021-06-01&$filter=policyType eq 'Custom'' Trying command 'ConvertFrom-Json -AsHashtable' [AzAPICall 1.2.1] 'Getting Policy definitions for Management Group: 'REDACTED' ('REDACTED')' uri='https://management.azure.com/providers/Microsoft.Management/managementgroups/REDACTED/providers/Microsoft.Authorization/policyDefinitions?api-version=2021-06-01&$filter=policyType eq 'Custom'' Command 'ConvertFrom-Json -AsHashtable' succeeded. Please file an issue at the AzGovViz GitHub repository (aka.ms/AzGovViz) and provide a dump (scrub subscription Id and company identifyable names) of the resource (portal JSOn view) - Thank you! [AzAPICall 1.2.1] 'Getting Policy definitions for Management Group: 'REDACTED' ('REDACTED')' uri='https://management.azure.com/providers/Microsoft.Management/managementgroups/REDACTED/providers/Microsoft.Authorization/policyDefinitions?api-version=2021-06-01&$filter=policyType eq 'Custom'' Command 'ConvertFrom-Json -AsHashtable' failed [AzAPICall 1.2.1] 'Getting Policy definitions for Management Group: 'REDACTED' ('REDACTED')' uri='https://management.azure.com/providers/Microsoft.Management/managementgroups/REDACTED/providers/Microsoft.Authorization/policyDefinitions?api-version=2021-06-01&$filter=policyType eq 'Custom'' Command 'ConvertFrom-Json -AsHashtable' failed. Please file an issue at the AzGovViz GitHub repository (aka.ms/AzGovViz) and provide a dump (scrub subscription Id and company identifyable names) of the resource (portal JSOn view) - Thank you! Exception: /agent/_work/1/s/Reporting/tmp/AzGovViz/pwsh/AzGovVizParallel.ps1:5733 Line | 5733 | $batchLevelGroupBatch | ForEach-Object -Parallel { | ~~~~~~~~~~ | throwing - Command ConvertFrom-Json -AsHashtable failed (different | casing)

JulianHayward commented 1 month ago

@ReneRebsdorf can you please, please share one of those policies? Is the existence of keys with different casing reasonable? Also have a look what the azAPICall is doing in that case: https://github.com/JulianHayward/AzAPICall/blob/5745ef736f7686e9f7e29a04fcf5b833945a0282/pwsh/module/dev/AzAPICall/functions/AzAPICall.ps1#L449

ReneRebsdorf commented 1 month ago

@JulianHayward

Found the policy with the problem, notice someone has mistakenly added the same property twice but with different casing. we have now removed the 'timegrain' property. If the error message had returned which policy definition with this fault it would have been easier to find :-) - I have redacted the user who created this back in 2021 as I do not find it in good taste to "point fingers" at someone who is not monitoring this issue :-) :

Closing issue.

{
  "properties": {
    "displayName": "Deploy Diagnostics Settings for Automation Account to Log Analytics Workspace",
    "policyType": "Custom",
    "mode": "All",
    "description": "Deploys the diagnostic settings for Automation Accounts to stream to a regional Log Analytics workspace.",
    "metadata": {
      "version": "1.0.0",
      "category": "Monitoring",
      "createdBy": "REDACTED-GUID",
      "createdOn": "2021-10-27T11:48:04.4676791Z",
      "updatedBy": "REDACTED-GUID",
      "updatedOn": "2022-11-02T13:02:06.3730134Z"
    },
    "version": "1.0.0",
    "parameters": {
      "logAnalytics": {
        "type": "String"
      }
    },
    "policyRule": {
      "if": {
        "field": "type",
        "equals": "Microsoft.Network/virtualNetworkGateways"
      },
      "then": {
        "effect": "deployIfNotExists",
        "details": {
          "type": "Microsoft.Insights/diagnosticSettings",
          "name": "setByPolicy",
          "roleDefinitionIds": [
            "/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
          ],
          "deployment": {
            "properties": {
              "mode": "incremental",
              "template": {
                "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                "contentVersion": "1.0.0.0",
                "parameters": {
                  "resourceName": {
                    "type": "string"
                  },
                  "logAnalytics": {
                    "type": "string"
                  },
                  "location": {
                    "type": "string"
                  }
                },
                "variables": {},
                "resources": [
                  {
                    "type": "Microsoft.Automation/automationAccounts/providers/diagnosticSettings",
                    "apiVersion": "2018-06-01",
                    "name": "DiagSettings",
                    "location": "[parameters('location')]",
                    "dependsOn": [],
                    "properties": {
                      "workspaceId": "[parameters('logAnalytics')]",
                      "metrics": [
                        {
                          "category": "AllMetrics",
                          "timegrain": null,
                          "enabled": false,
                          "retentionPolicy": {
                            "days": 0,
                            "enabled": false
                          },
                          "timeGrain": null
                        }
                      ],
                      "logs": [
                        {
                          "category": "JobLogs",
                          "enabled": true
                        },
                        {
                          "category": "JobStreams",
                          "enabled": true
                        },
                        {
                          "category": "DscNodeStatus",
                          "enabled": true
                        }
                      ]
                    }
                  }
                ],
                "outputs": {}
              },
              "parameters": {
                "logAnalytics": {
                  "value": "[parameters('logAnalytics')]"
                },
                "location": {
                  "value": "[field('Location')]"
                },
                "resourceName": {
                  "value": "[field('Name')]"
                }
              }
            }
          }
        }
      }
    },
    "versions": [
      "1.0.0"
    ]
  },
  "id": "/providers/Microsoft.Management/managementGroups/REDACTED/providers/Microsoft.Authorization/policyDefinitions/enableAutomationAccountDiagSetting",
  "type": "Microsoft.Authorization/policyDefinitions",
  "name": "enableAutomationAccountDiagSetting",
  "systemData": {
    "createdBy": "REDACTED",
    "createdByType": "User",
    "createdAt": "2021-10-27T11:48:04.4212543Z",
    "lastModifiedBy": "7797d39c-c6ae-4621-96c9-723ce2ec680f",
    "lastModifiedByType": "Application",
    "lastModifiedAt": "2022-11-02T13:02:05.1973217Z"
  }
}