az cosmosdb keys list returns Operation returned an invalid status 'Bad Request'

[ezYakaEagle442]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command

Describe the bug

az cosmosdb keys list command fails with:

Operation returned an invalid status 'Bad Request'

To Reproduce

          cosmos_account_name=$(az deployment group show --name cosmos-db  -g ${{ env.RG_APP }} --query properties.outputs.cosmosAccountName.value -o tsv)
          echo "cosmos_account_name" $cosmos_account_name
          COSMOS_DB_KEY1=$(az cosmosdb keys list --name $cosmos_account_name -g ${{ env.RG_APP }} --query "primaryMasterKey" | tr -d '"')
          COSMOS_DB_KEY2=$(az cosmosdb keys list --name $cosmos_account_name -g ${{ env.RG_APP }} --query "secondaryMasterKey" | tr -d '"')

Expected behavior

The command should run successfully, strangely as a workaround the command runs successfully when I give a hardcoded value of the cosmos_account_name :

az cosmosdb keys list --name cosmos-XXXXXX-poc  -g rg-aca-XXXXXX

{
  "primaryMasterKey": "AhAhAh424242ZZZ",
  "primaryReadonlyMasterKey": "AhAhAh424242ZZZ",
  "secondaryMasterKey": "AhAhAh424242ZZZ",
  "secondaryReadonlyMasterKey": "AhAhAh424242ZZZ"
}

Environment summary

VSCode/Terminal/WSL2/Ubuntu2204

 az version
{
  "azure-cli": "2.45.0",
  "azure-cli-core": "2.45.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "spring": "1.6.8"
  }
}

Additional context

[yonzhan]

route to CXP team

[PramodValavala-MSFT]

@ezYakaEagle442 I was unable to reproduce this. Could you run this command with the --debug flag and share the output here?

[ezYakaEagle442]

cli.knack.cli: Command arguments: ['cosmosdb', 'keys', 'list', '--name', 'cosmos-reddog7wwufb7rkg2oe-poc\r', '-g', 'rg-aca-reddog-quarkus', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x03C7A418>, <function OutputProducer.on_global_arguments at 0x03F78BB0>, <f
unction CLIQuery.on_global_arguments at 0x03F95808>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'cosmosdb': ['azure.cli.command_modules.cosmosdb']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: cosmosdb                  0.016        52       184
cli.azure.cli.core: Total (1)                 0.016        52       184
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: Total (0)                 0.000         0         0
cli.azure.cli.core: Loaded 52 groups, 184 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : cosmosdb keys list
cli.azure.cli.core: Command table: cosmosdb keys list
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x04A37DF0>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\bob\.azure\commands\2023-03-03.09-59-55.cosmosdb_keys_list.46528.log'.
az_command_data_logger: command args: cosmosdb keys list --name {} -g {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x04A5DFA0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x04A630B8>, <function register_cache_arguments.<locals>
.add_cache_arguments at 0x04A6E610>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x04A871D8>]
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x03F78BF8>, <function CLIQuery.handle_query_parameter at 0x03F95850>, <func
tion register_ids_argument.<locals>.parse_ids_arguments at 0x04A6E5C8>]
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=CosmosDBManagementClient
cli.azure.cli.core.auth.persistence: build_persistence: location='C:\\Users\\bob\\.azure\\msal_token_cache.bin', encrypt=True
cli.azure.cli.core.auth.binary_cache: load: C:\Users\bob\.azure\msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/42424242424242/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported
': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/42424242424242/discovery/v2.0/keys', 'respo
nse_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['
code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/7744e593-db
09-4d56-bcbd-c874c9323670/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login
.microsoftonline.com/42424242424242/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/7744e593-db09-4d56-bcbd-c874c93
23670/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/7744e593-db09-4d56-bc
bd-c874c9323670/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'ia
t', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/42424242424242/kerberos', 'tenant_region_scope': 'NA', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.micros
oft.com', 'rbac_url': 'https://pas.windows.net'}
msal.application: Broker enabled? False
cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), claims=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: 652bbbc8-76d0-433f-8d51-3325bbcafc3a
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/212121212121/resourceGroups/rg-aca-reddog-quarkus/providers/Microso
ft.DocumentDB/databaseAccounts/cosmos-reddog7wwufb7rkg2oe-poc%0D/listKeys?api-version=2022-08-15'
cli.azure.cli.core.sdk.policies: Request method: 'POST'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies:     'Accept': 'application/json'
cli.azure.cli.core.sdk.policies:     'x-ms-client-request-id': 'c466d861-b9a1-11ed-9d10-a02942ae5ee7'
cli.azure.cli.core.sdk.policies:     'CommandName': 'cosmosdb keys list'
cli.azure.cli.core.sdk.policies:     'ParameterSetName': '--name -g --debug'
cli.azure.cli.core.sdk.policies:     'User-Agent': 'AZURECLI/2.45.0 azsdk-python-mgmt-cosmosdb/8.0.0 Python/3.10.8 (Windows-10-10.0.22621-SP0)'
cli.azure.cli.core.sdk.policies:     'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: This request has no body
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "POST /subscriptions/212121212121/resourceGroups/rg-aca-reddog-quarkus/providers/Microsoft.DocumentD
B/databaseAccounts/cosmos-reddog7wwufb7rkg2oe-poc%0D/listKeys?api-version=2022-08-15 HTTP/1.1" 400 324
cli.azure.cli.core.sdk.policies: Response status: 400
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies:     'Content-Type': 'text/html; charset=us-ascii'
cli.azure.cli.core.sdk.policies:     'Server': 'Microsoft-HTTPAPI/2.0'
cli.azure.cli.core.sdk.policies:     'Date': 'Fri, 03 Mar 2023 08:59:57 GMT'
cli.azure.cli.core.sdk.policies:     'Connection': 'close'
cli.azure.cli.core.sdk.policies:     'Content-Length': '324'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/cosmosdb/custom.py", line 478, in cli_cosmosdb_keys
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/core/tracing/decorator.py", line 73, in wrapper_use_tracer
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/cosmosdb/operations/_database_accounts_operations.py", line 1688, in list_keys
azure.core.exceptions.HttpResponseError: Operation returned an invalid status 'Bad Request'
Content: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

cli.azure.cli.core.azclierror: Operation returned an invalid status 'Bad Request'
az_command_data_logger: Operation returned an invalid status 'Bad Request'
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x04A37F10>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 0.809 seconds (init: 0.372, invoke: 0.437)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3262 in cache
telemetry.check: Negative: The C:\Users\bob\.azure\telemetry.txt was modified at 2023-03-03 09:59:01.242364, which in less than 600.000000 s

[ezYakaEagle442]

what the hell is that File "D:\a_work\1\s\build_scripts\windows\xxxxx ..... ??

[ezYakaEagle442]

@ezYakaEagle442 I was unable to reproduce this. Could you run this command with the --debug flag and share the output here?

Can you please try not on Windows, on a GitHub Workflow and VSCode/Terminal/WSL2/Ubuntu2204 ? (as there is no file path D:\a_work\1\s\build_scripts\windows\xxxxx .....)

[ezYakaEagle442]

@PramodValavala-MSFT I share with you the Bicep template so that yo ucan test :

@description('A UNIQUE name')
@maxLength(33)
param appName string = 'cosmos-reddog${uniqueString(resourceGroup().id, subscription().id)}-poc' // cosmos-<project, app or service>-<environment>

param location string = resourceGroup().location

param cosmosAccountName string = 'cosmos-${appName}-poc' // cosmos-<project, app or service>-<environment>
param cosmosDatabaseName string = 'reddog'
param cosmosCollectionName string = 'reddog'

@allowed([ 'GlobalDocumentDB', 'MongoDB', 'Parse' ])
param cosmosAccountKind string = 'MongoDB'

param containers array = []
param databaseName string

@description('The list of capabilities for the Cosmos DB account? See https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/how-to-configure-capabilities#available-capabilities : DisableRateLimitingResponses, EnableMongoRoleBasedAccessControl, EnableMongoRetryableWrites, EnableMongo16MBDocumentSupport, EnableUniqueCompoundNestedDocs')
param capabilities array = [ { name: 'EnableMongoRoleBasedAccessControl' } ]

// https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/allversions
resource cosmosAccount 'Microsoft.DocumentDB/databaseAccounts@2022-08-15' = {
  name: cosmosAccountName
  kind: cosmosAccountKind
  location: location
  properties: {
    consistencyPolicy: {
      defaultConsistencyLevel: 'Session'
    }
    locations: [
      {
        locationName: location
      }
    ]
    databaseAccountOfferType: 'Standard'
    enableAutomaticFailover: false
    enableMultipleWriteLocations: false
    capabilities: capabilities
  }
}

output cosmosAccountResourceId string = cosmosAccount.id
output cosmosUri string = 'https://${cosmosAccountName}.documents.azure.com:443/'
output cosmosAccountName string = cosmosAccount.name
output cosmosDocumentEndpoint string = cosmosAccount.properties.documentEndpoint

//output cosmosAccountXXX string = cosmosAccount.listConnectionStrings()
//utput cosmosAccountXXX string = cosmosAccount.listKeys()

resource cosmosDatabase 'Microsoft.DocumentDB/databaseAccounts/mongodbDatabases@2022-08-15' = {
  parent: cosmosAccount
  name: cosmosDatabaseName
  properties: {
    resource: {
      id: cosmosDatabaseName
    }
  }
}

output cosmosDatabaseResourceId string = cosmosDatabase.id
output cosmosDatabaseName string = cosmosDatabase.name

resource cosmosCollection 'Microsoft.DocumentDB/databaseAccounts/mongodbDatabases/collections@2022-08-15' = {
  parent: cosmosDatabase
  name: cosmosCollectionName
  properties: {
    resource: {
      id: cosmosCollectionName
      shardKey: {
        _id: 'Hash'
      }
    }
  }
}

output cosmosCollectionId string = cosmosCollection.properties.resource.id
output cosmosCollectionName string = cosmosCollection.name

/*
resource database 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases@2022-08-15' = {
  name: '${cosmosAccountName}/${databaseName}'
  properties: {
    resource: { id: databaseName }
  }

resource list 'containers' = [for container in containers: {
  name: container.name
  properties: {
    resource: {
      id: container.id
      partitionKey: { paths: [ container.partitionKey ] }
    }
    options: {}
  }
}]
dependsOn: [
  cosmosAccount
]
}
*/

[ezYakaEagle442]

I have tested and this issue does not happen running CLI on Windows :

FOR /F %i IN ('az deployment group show --name cosmos-db  -g rg-aca-reddog-quarkus --query properties.outputs.cosmosAccountName.value -o tsv') DO set cosmos_account_name=%i
az cosmosdb keys list --name %cosmos_account_name% -g rg-aca-reddog-quarkus 

So this there is definitely a bug in CLI when running on Linux, it add a Carriage Return '0D' when running 'az deployment group show'

[ezYakaEagle442]

trim '\r' is a workaround :

cosmos_account_name=$(az deployment group show --name cosmos-db -g rg-aca-reddog-quarkus --query properties.outputs.cosmosAccountName.value -o tsv | tr -d '\r') echo "cosmos_account_name" $cosmos_account_name

[PramodValavala-MSFT]

@ezYakaEagle442 Just gave this a shot in the Azure CLI docker container and couldn't reproduce it still. Could you try to reproduce the same in the Azure CLI docker container and Azure Cloud Shell as well?

Also, do you see the \r when running the az deployment show command with the --debug flag as well?

Making a guess here but are you running these commands as-is or are they part of a script file? If they are, then it could be best to check/convert the line endings of the file to LF from CRLF in your editor, or use dos2unisx.

[ezYakaEagle442]

indeed my bad, I did get a new laptop, reinattled VSCode which by default set \r everywhere :(

To fix it, I have done :

Edit parameters Cog / Settings / Search for 'files.eol' and set '\n'

To remove '\r'

find . -type f -print0 | xargs -0 dos2unix --