Azure / azure-rest-api-specs

The source for REST API specifications for Microsoft Azure.
MIT License
2.67k stars 5.09k forks source link

`Microsoft.Insights/diagnosticSettings@2021-05-01-preview` - `logAnalyticsDestinationType` break PUT-GET consistency #22400

Open teowa opened 1 year ago

teowa commented 1 year ago

Background

Under some situation when set logAnalyticsDestinationType as Dedicated, the API returns null, and when set logAnalyticsDestinationType as null (or not set), the API returns AzureDiagnostics. This breaks PUT-GET consistency, see roundtrip_inconsistent_property. The diagnostic setting can be used for many target resource, it is hard to fully test all the situation. Below shows two cases:

  1. target KeyVault, PUT AzureDiagnostics or Dedicated, GET null
  2. target DataFactory (V2), PUT null, GET AzureDiagnostics. But PUT AzureDiagnostics or Dedicated can work well.

swagger:

https://github.com/Azure/azure-rest-api-specs/blob/e0a8a42064db5ad298a3f660805d65283c07aca2/specification/monitor/resource-manager/Microsoft.Insights/preview/2021-05-01-preview/diagnosticsSettings_API.json#L323-L327

API flow 1, target KeyVault, PUT AzureDiagnostics or Dedicated, GET null

PUT https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.KeyVault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp?api-version=2021-05-01-preview x-ms-request-id: 34a48570-6b21-49fd-af95-f611c88ec97f x-ms-correlation-request-id: 04fe903c-847d-427c-b23b-9ed8c53966b9

payload ```json { "properties": { "logs": [ { "category": "AuditEvent", "categoryGroup": null, "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } }, { "category": "AzurePolicyEvaluationDetails", "categoryGroup": null, "enabled": false, "retentionPolicy": { "days": 0, "enabled": false } } ], "metrics": [ { "timeGrain": null, "enabled": true, "retentionPolicy": { "days": 0, "enabled": false }, "category": "AllMetrics" } ], "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw", "logAnalyticsDestinationType": "AzureDiagnostics" } } ``` 200 OK ```json { "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.keyvault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp", "type": "Microsoft.Insights/diagnosticSettings", "name": "exp", "location": null, "kind": null, "tags": null, "properties": { "storageAccountId": null, "serviceBusRuleId": null, "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw", "eventHubAuthorizationRuleId": null, "eventHubName": null, "metrics": [ { "timeGrain": "PT1M", "category": "AllMetrics", "enabled": true, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logs": [ { "category": "AuditEvent", "categoryGroup": null, "enabled": true, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "AzurePolicyEvaluationDetails", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logAnalyticsDestinationType": "AzureDiagnostics" }, "identity": null } ```

GET https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.KeyVault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp?api-version=2021-05-01-preview x-ms-request-id: 1cb12d3e-463e-4a4d-9a8c-d1c963a2c4e3 x-ms-correlation-request-id: f430d3e4-c539-4cc5-b4be-78d3585e7322 200 OK

payload ```json { "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.keyvault/vaults/wtexpkeyvault/providers/microsoft.insights/diagnosticSettings/exp", "type": "Microsoft.Insights/diagnosticSettings", "name": "exp", "location": null, "kind": null, "tags": null, "properties": { "storageAccountId": null, "serviceBusRuleId": null, "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw", "eventHubAuthorizationRuleId": null, "eventHubName": null, "metrics": [ { "category": "AllMetrics", "enabled": true, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logs": [ { "category": "AuditEvent", "categoryGroup": null, "enabled": true, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "AzurePolicyEvaluationDetails", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logAnalyticsDestinationType": null }, "identity": null } ```

API flow 2, target DataFactory (V2), PUT null, GET AzureDiagnostics. But PUT AzureDiagnostics or Dedicated can work well

PUT https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.DataFactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS?api-version=2021-05-01-preview x-ms-request-id: d738f84e-cf08-449a-a370-7b328ed7e3bd x-ms-correlation-request-id: f65996b6-3bf5-492f-bf1c-b6a7734d8bdd

payload ```json { "properties": { "logs": [ { "category": "SSISIntegrationRuntimeLogs", "categoryGroup": null, "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } } ], "metrics": [ { "timeGrain": null, "enabled": false, "retentionPolicy": { "days": 0, "enabled": false }, "category": "AllMetrics" } ], "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw", "logAnalyticsDestinationType": "" } } ``` 200 OK ```json { "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.datafactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS", "type": "Microsoft.Insights/diagnosticSettings", "name": "wt-datafac-DS", "location": null, "kind": null, "tags": null, "properties": { "storageAccountId": null, "serviceBusRuleId": null, "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw", "eventHubAuthorizationRuleId": null, "eventHubName": null, "metrics": [ { "timeGrain": "PT1M", "category": "AllMetrics", "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logs": [ { "category": "SSISIntegrationRuntimeLogs", "categoryGroup": null, "enabled": true, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logAnalyticsDestinationType": "" }, "identity": null } ```

GET https://management.azure.com/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.DataFactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS?api-version=2021-05-01-preview

x-ms-request-id: f667dc88-c6e5-4eb1-ab12-e4685b271b42 x-ms-correlation-request-id: b438f808-33c0-4d7e-b93a-d1d82b4581a2 200 OK

payload ```json { "id": "/subscriptions/XXXX/resourcegroups/wt-diagnostic-resources/providers/microsoft.datafactory/factories/wt-acctest/providers/microsoft.insights/diagnosticSettings/wt-datafac-DS", "type": "Microsoft.Insights/diagnosticSettings", "name": "wt-datafac-DS", "location": null, "kind": null, "tags": null, "properties": { "storageAccountId": null, "serviceBusRuleId": null, "workspaceId": "/subscriptions/XXXX/resourceGroups/wt-diagnostic-resources/providers/Microsoft.OperationalInsights/workspaces/wt-testlaw", "eventHubAuthorizationRuleId": null, "eventHubName": null, "metrics": [ { "category": "AllMetrics", "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logs": [ { "category": "SSISIntegrationRuntimeLogs", "categoryGroup": null, "enabled": true, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "ActivityRuns", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "PipelineRuns", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "TriggerRuns", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "SandboxPipelineRuns", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "SandboxActivityRuns", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "SSISPackageEventMessages", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "SSISPackageExecutableStatistics", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "SSISPackageEventMessageContext", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "SSISPackageExecutionComponentPhases", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } }, { "category": "SSISPackageExecutionDataStatistics", "categoryGroup": null, "enabled": false, "retentionPolicy": { "enabled": false, "days": 0 } } ], "logAnalyticsDestinationType": "AzureDiagnostics" }, "identity": null } ```
LaurentLesle commented 5 months ago

Impacting the Azure Verified Module project -> https://github.com/Azure/terraform-azurerm-avm-res-kusto-cluster/actions/runs/9090952291/job/24984653056?pr=27#step:3:228

We are using AzureDedicated to the destination type on a Kusto Explorer. Fun fact: the diagnostic profiles are sent to ADX* tables so the PUT is working but the get is null forcing Terraform to re-modify the service. AVM has a rule to get modules idempotent and we cannot achieve it dur to this issue.

Need a fix in the API to get a fix in the golang SDK to fix Terraform Azurerm.