[BUG] Microsoft.DocumentDB/databaseAccounts - GET DatabaseAccount : sqlEndpoint, configurationOverrides -- additional properties

dj-r1 commented 6 days ago

API Spec link

https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cosmos-db/resource-manager/Microsoft.DocumentDB/preview/2024-02-15-preview/cosmos-db.json

API Spec version

2024-02-15-preview, 2023-11-15

Describe the bug

Operation DatabaseAccounts_Get in cosmos-db.json, so simple GET DatabaseAccount

As example CosmosDBDatabaseAccountGet.json, response describing a resource, it should be without: properties.sqlEndpoint properties.configurationOverrides: {...} properties.defaultPriorityLevel properties.enablePerRegionPerPartitionAutoscale properties.enablePriorityBasedExecution minMaxThresholdsForPriorityBasedExecution: {...}

There are also examples in Azure REST API for Database Accounts - Get. Found with Microsoft.DocumentDB ARM template/Bicep description.

Using it with Azure Cosmos DB for MongoDB account (RU).

Expected behavior

{
  "id": "/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.DocumentDB/databaseAccounts/my-cosmosdb",
  "identity": {
    "type": "None"
  },
  "kind": "MongoDB",
  "location": "Europe",
  "name": "my-cosmosdb",
  "properties": {
    "EnabledApiTypes": "MongoDB",
    "analyticalStorageConfiguration": {
      "schemaType": "FullFidelity"
    },
    "apiProperties": {
      "serverVersion": "4.2"
    },
    "backupPolicy": {
      "periodicModeProperties": {
        "backupIntervalInMinutes": 240,
        "backupRetentionIntervalInHours": 8,
        "backupStorageRedundancy": "Geo"
      },
      "type": "Periodic"
    },
    "capabilities": [
      {
        "name": "EnableMongo"
      }
    ],
    "configurationOverrides": {
      "EnableBsonSchema": "True",
      "EnablePerRegionPerPartitionAutoscaleOptIn": "True"
    },
    "consistencyPolicy": {
      "defaultConsistencyLevel": "Eventual",
      "maxIntervalInSeconds": 5,
      "maxStalenessPrefix": 100
    },
    "cors": [],
    "databaseAccountOfferType": "Standard",
    "defaultIdentity": "FirstPartyIdentity",
    "diagnosticLogSettings": {
      "enableFullTextQuery": "None"
    },
    "disableKeyBasedMetadataWriteAccess": true,
    "disableLocalAuth": false,
    "documentEndpoint": "https://my-cosmosdb.documents.azure.com:443/",
    "enableAnalyticalStorage": true,
    "enableAutomaticFailover": false,
    "enableBurstCapacity": false,
    "enableFreeTier": false,
    "enableMultipleWriteLocations": false,
    "enablePartitionKeyMonitor": false,
    "enablePartitionMerge": false,
    "failoverPolicies": [
      {
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "locationName": "Europe"
      }
    ],
    "instanceId": "1111bb22-22aa-1a1a-11a1-1111aaa1a111",
    "ipRules": [
      {
        "ipAddressOrRange": "104.1.1.1"
      }
    ],
    "isVirtualNetworkFilterEnabled": true,
    "keysMetadata": {
      "primaryMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      },
      "primaryReadonlyMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      },
      "secondaryMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      },
      "secondaryReadonlyMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      }
    },
    "locations": [
      {
        "documentEndpoint": "https://my-cosmosdb-europe.documents.azure.com:443/",
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "isZoneRedundant": false,
        "locationName": "Europe",
        "provisioningState": "Succeeded"
      }
    ],
    "minimalTlsVersion": "Tls12",
    "mongoEndpoint": "https://my-cosmosdb.mongo.cosmos.azure.com:443/",
    "networkAclBypass": "None",
    "networkAclBypassResourceIds": [],
    "privateEndpointConnections": [
      {
        "id": "/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.DocumentDB/databaseAccounts/my-cosmosdb/privateEndpointConnections/my-cosmosdb-MongoDB-pe",
        "properties": {
          "privateEndpoint": {
            "id": "/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.Network/privateEndpoints/my-cosmosdb-MongoDB-pe"
          },
          "privateLinkServiceConnectionState": {
            "actionsRequired": "None",
            "status": "Approved"
          }
        }
      }
    ],
    "provisioningState": "Succeeded",
    "publicNetworkAccess": "Disabled",
    "readLocations": [
      {
        "documentEndpoint": "https://my-cosmosdb-europe.documents.azure.com:443/",
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "isZoneRedundant": false,
        "locationName": "Europe",
        "provisioningState": "Succeeded"
      }
    ],
    "virtualNetworkRules": [],
    "writeLocations": [
      {
        "documentEndpoint": "https://my-cosmosdb-europe.documents.azure.com:443/",
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "isZoneRedundant": false,
        "locationName": "Europe",
        "provisioningState": "Succeeded"
      }
    ]
  },
  "systemData": {
    "createdAt": "2023-12-01T11:59:35.4483112+00:00"
  },
  "tags": {},
  "type": "Microsoft.DocumentDB/databaseAccounts"
}

Actual behavior

{
  "id": "/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.DocumentDB/databaseAccounts/my-cosmosdb",
  "identity": {
    "type": "None"
  },
  "kind": "MongoDB",
  "location": "Europe",
  "name": "my-cosmosdb",
  "properties": {
    "EnabledApiTypes": "MongoDB",
    "analyticalStorageConfiguration": {
      "schemaType": "FullFidelity"
    },
    "apiProperties": {
      "serverVersion": "4.2"
    },
    "backupPolicy": {
      "periodicModeProperties": {
        "backupIntervalInMinutes": 240,
        "backupRetentionIntervalInHours": 8,
        "backupStorageRedundancy": "Geo"
      },
      "type": "Periodic"
    },
    "capabilities": [
      {
        "name": "EnableMongo"
      }
    ],
    "consistencyPolicy": {
      "defaultConsistencyLevel": "Eventual",
      "maxIntervalInSeconds": 5,
      "maxStalenessPrefix": 100
    },
    "cors": [],
    "databaseAccountOfferType": "Standard",
    "defaultIdentity": "FirstPartyIdentity",
    "defaultPriorityLevel": "High",
    "diagnosticLogSettings": {
      "enableFullTextQuery": "None"
    },
    "disableKeyBasedMetadataWriteAccess": true,
    "disableLocalAuth": false,
    "documentEndpoint": "https://my-cosmosdb.documents.azure.com:443/",
    "enableAnalyticalStorage": true,
    "enableAutomaticFailover": false,
    "enableBurstCapacity": false,
    "enableFreeTier": false,
    "enableMultipleWriteLocations": false,
    "enablePartitionKeyMonitor": false,
    "enablePartitionMerge": false,
    "enablePerRegionPerPartitionAutoscale": false,
    "enablePriorityBasedExecution": false,
    "failoverPolicies": [
      {
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "locationName": "Europe"
      }
    ],
    "instanceId": "1111bb22-22aa-1a1a-11a1-1111aaa1a111",
    "ipRules": [
      {
        "ipAddressOrRange": "104.1.1.1"
      }
    ],
    "isVirtualNetworkFilterEnabled": true,
    "keysMetadata": {
      "primaryMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      },
      "primaryReadonlyMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      },
      "secondaryMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      },
      "secondaryReadonlyMasterKey": {
        "generationTime": "2023-12-01T11:59:35.4483112+00:00"
      }
    },
    "locations": [
      {
        "documentEndpoint": "https://my-cosmosdb-europe.documents.azure.com:443/",
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "isZoneRedundant": false,
        "locationName": "Europe",
        "provisioningState": "Succeeded"
      }
    ],
    "minMaxThresholdsForPriorityBasedExecution": {
      "maxPercentForHighPriorityRequests": 100,
      "maxPercentForLowPriorityRequests": 100,
      "minPercentForHighPriorityRequests": 0,
      "minPercentForLowPriorityRequests": 0
    },
    "minimalTlsVersion": "Tls12",
    "mongoEndpoint": "https://my-cosmosdb.mongo.cosmos.azure.com:443/",
    "networkAclBypass": "None",
    "networkAclBypassResourceIds": [],
    "privateEndpointConnections": [
      {
        "id": "/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.DocumentDB/databaseAccounts/my-cosmosdb/privateEndpointConnections/my-cosmosdb-MongoDB-pe",
        "properties": {
          "privateEndpoint": {
            "id": "/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.Network/privateEndpoints/my-cosmosdb-MongoDB-pe"
          },
          "privateLinkServiceConnectionState": {
            "actionsRequired": "None",
            "status": "Approved"
          }
        }
      }
    ],
    "provisioningState": "Succeeded",
    "publicNetworkAccess": "Disabled",
    "readLocations": [
      {
        "documentEndpoint": "https://my-cosmosdb-europe.documents.azure.com:443/",
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "isZoneRedundant": false,
        "locationName": "Europe",
        "provisioningState": "Succeeded"
      }
    ],
    "sqlEndpoint": "https://my-cosmosdb.documents.azure.com:443/",
    "virtualNetworkRules": [],
    "writeLocations": [
      {
        "documentEndpoint": "https://my-cosmosdb-europe.documents.azure.com:443/",
        "failoverPriority": 0,
        "id": "my-cosmosdb-europe",
        "isZoneRedundant": false,
        "locationName": "Europe",
        "provisioningState": "Succeeded"
      }
    ]
  },
  "systemData": {
    "createdAt": "2023-12-01T11:59:35.4483112+00:00"
  },
  "tags": {},
  "type": "Microsoft.DocumentDB/databaseAccounts"
}

Reproduction Steps

az rest --method get --header "Accept=application/json" --url 'https://management.azure.com/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.DocumentDB/databaseAccounts/my-cosmosdb?api-version=2024-02-15-preview' || GET https://management.azure.com/subscriptions/aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb/resourceGroups/my-rg/providers/Microsoft.DocumentDB/databaseAccounts/my-cosmosdb?api-version=2024-02-15-preview

also with api-version=2023-11-15

Environment

No response

v-jiaodi commented 5 days ago

@carjackson-msft Please help take a look, thanks.

carjackson-msft commented 5 days ago

Hi @v-jiaodi - in the future, can you please assign these tickets directly to @markjbrown for triaging? I have re-assigned accordingly. CC: @pjohari-ms

markjbrown commented 5 days ago

I have no idea what the question is here. @dj-r1 can you clarify?

dj-r1 commented 5 days ago

@markjbrown I raise the issue / bug that properties sqlEndpoint, configurationOverrides, defaultPriorityLevel, enablePerRegionPerPartitionAutoscale, enablePriorityBasedExecution, minMaxThresholdsForPriorityBasedExecution shouldn't be included in the response of Azure REST API Database Accounts - Get (Microsoft.DocumentDB) as not defined (in DatabaseAccountGetResults).

markjbrown commented 4 days ago

The API docs for our resource provider are autogenerated from our swagger spec. The example output you see is taken from examples/CosmosDBDatabaseAccountGet.json file where our swagger spec is.

Within the swagger spec itself, definition for DatabaseAccountGetProperties also does not list every possible value that can be returned so it also would not be of much help here.

@carjackson-msft and @pjohari-ms, I'd like to get your thoughts on this. Historically, we've not really been exhaustive on documenting these and I've never seen asks from customers to provide explicit api docs/examples on these.

Not sure if there is a low-friction way to update the api-version examples via unit tests. Or if/how the resource definitions in the swagger spec can be updated to provide exhaustive details on return values.

Thanks.

dj-r1 commented 9 hours ago

I'm trying to understand if these fields are valid for API. Based on @markjbrown comment, I may presume that ARM-schemas in their Microsoft.DocumentDB.json relied on not complete API spec, or intentionally haven't specified these fields, have they? I see it as a chain of dependencies: Azure-rest-api-specs → ARM template → Bicep → Bicep/ARM WhatIf → me. API returns them to me, so I'd like to identify where it should be suppressed or covered/specified.

Azure / azure-rest-api-specs