search

Azure / azure-cli

Azure Command-Line Interface
MIT License
3.96k stars 2.94k forks source link

az redis list-keys command fails with "Operation returned an invalid status 'Bad Request' " #25648

Closed ezYakaEagle442 closed 1 year ago

ezYakaEagle442 commented 1 year ago

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command

az redis list-keys

Describe the bug

az redis list-keys command fails

redis_name=$(az deployment group show --name redis  -g ${{ env.RG_APP }} --query properties.outputs.redisName.value -o tsv)
 echo "redis_name" $redis_name

 az redis list-keys -n $redis_name -g rg-aca-XXX
Operation returned an invalid status 'Bad Request'

To Reproduce

 redis_name=$(az deployment group show --name redis  -g ${{ env.RG_APP }} --query properties.outputs.redisName.value -o tsv)
echo "redis_name" $redis_name

redis_id=$(az deployment group show --name redis  -g ${{ env.RG_APP }} --query properties.outputs.redisResourceId.value -o tsv)
echo "redis_id" $redis_id

REDIS_KEY1=$(az redis list-keys --ids $redis_id --query "primaryKey" | tr -d '"')
REDIS_KEY2=$(az redis list-keys --ids $redis_id --query "secondaryKey" | tr -d '"')

echo $REDIS_KEY1 # this returns correctly the Key
echo $REDIS_KEY2 # this returns correctly the Key

 az redis list-keys -n $redis_name -g RG_APP # this fails

Expected behavior

az redis list-keys --name command should run sucessfully

Environment summary

SCode/Terminal/WSL2/Ubunti2204

 az version
{
  "azure-cli": "2.45.0",
  "azure-cli-core": "2.45.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "spring": "1.6.8"
  }
}

Additional context

yonzhan commented 1 year ago

route to CXP team

PramodValavala-MSFT commented 1 year ago

@ezYakaEagle442 I was unable to reproduce this error. Could you run the command with the --debug flag and share the output?

ezYakaEagle442 commented 1 year ago 
cli.knack.cli: Command arguments: ['redis', 'list-keys', '-n', 'redis-reddog7wwufb7rkg2oe-poc\r', '-g', 'rg-aca-reddog-quarkus', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x019DA418>, <function OutputProducer.on_global_arguments at 0x03498BB0>, <f
unction CLIQuery.on_global_arguments at 0x034B5808>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'redis': ['azure.cli.command_modules.redis']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: redis                     0.013         5        27
cli.azure.cli.core: Total (1)                 0.013         5        27
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: Total (0)                 0.000         0         0
cli.azure.cli.core: Loaded 5 groups, 27 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : redis list-keys
cli.azure.cli.core: Command table: redis list
cli.azure.cli.core: remaining    :            keys
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x04017DF0>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\bob\.azure\commands\2023-03-03.10-04-30.redis_list-keys.43148.log'.
az_command_data_logger: command args: redis list-keys -n {} -g {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x0403DFA0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x040430B8>, <function register_cache_arguments.<locals>
.add_cache_arguments at 0x0404E610>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x03498BF8>, <function CLIQuery.handle_query_parameter at 0x034B5850>, <func
tion register_ids_argument.<locals>.parse_ids_arguments at 0x0404E5C8>]
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=RedisManagementClient
cli.azure.cli.core.auth.persistence: build_persistence: location='C:\\Users\\bob\\.azure\\msal_token_cache.bin', encrypt=True
cli.azure.cli.core.auth.binary_cache: load: C:\Users\bob\.azure\msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/212121212121/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported
': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/212121212121/discovery/v2.0/keys', 'respo
nse_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['
code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/7744e593-db
09-4d56-bcbd-c874c9323670/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login
.microsoftonline.com/212121212121/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/7744e593-db09-4d56-bcbd-c874c93
23670/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/7744e593-db09-4d56-bc
bd-c874c9323670/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'ia
t', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/212121212121/kerberos', 'tenant_region_scope': 'NA', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.micros
oft.com', 'rbac_url': 'https://pas.windows.net'}
msal.application: Broker enabled? False
cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), claims=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: 42351d6f-1afc-4b85-98bd-ff1c6641316d
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/212121212121/resourceGroups/rg-aca-reddog-quarkus/providers/Microso
ft.Cache/redis/redis-reddog7wwufb7rkg2oe-poc%0D/listKeys?api-version=2022-06-01'
cli.azure.cli.core.sdk.policies: Request method: 'POST'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies:     'Accept': 'application/json'
cli.azure.cli.core.sdk.policies:     'x-ms-client-request-id': '6812a61f-b9a2-11ed-8ed1-a02942ae5ee7'
cli.azure.cli.core.sdk.policies:     'CommandName': 'redis list-keys'
cli.azure.cli.core.sdk.policies:     'ParameterSetName': '-n -g --debug'
cli.azure.cli.core.sdk.policies:     'User-Agent': 'AZURECLI/2.45.0 azsdk-python-mgmt-redis/14.1.0 Python/3.10.8 (Windows-10-10.0.22621-SP0)'
cli.azure.cli.core.sdk.policies:     'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: This request has no body
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "POST /subscriptions/212121212121/resourceGroups/rg-aca-reddog-quarkus/providers/Microsoft.Cache/red
is/redis-reddog7wwufb7rkg2oe-poc%0D/listKeys?api-version=2022-06-01 HTTP/1.1" 400 324
cli.azure.cli.core.sdk.policies: Response status: 400
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies:     'Content-Type': 'text/html; charset=us-ascii'
cli.azure.cli.core.sdk.policies:     'Server': 'Microsoft-HTTPAPI/2.0'
cli.azure.cli.core.sdk.policies:     'Date': 'Fri, 03 Mar 2023 09:04:31 GMT'
cli.azure.cli.core.sdk.policies:     'Connection': 'close'
cli.azure.cli.core.sdk.policies:     'Content-Length': '324'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

azure.mgmt.redis._serialization: Ran into a deserialization error. Ignoring since this is failsafe deserialization
Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/_serialization.py", line 1433, in _deserialize
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/_serialization.py", line 1180, in rest_key_extractor
AttributeError: 'str' object has no attribute 'get'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/_serialization.py", line 1506, in failsafe_deserialize
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/_serialization.py", line 1378, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/_serialization.py", line 1448, in _deserialize
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/core/exceptions.py", line 78, in raise_with_traceback
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/_serialization.py", line 1433, in _deserialize
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/_serialization.py", line 1180, in rest_key_extractor
azure.core.exceptions.DeserializationError: (", AttributeError: 'str' object has no attribute 'get'", 'Unable to deserialize to object: type', AttributeError("'str' object has no at
tribute 'get'"))
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/core/tracing/decorator.py", line 73, in wrapper_use_tracer
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/mgmt/redis/operations/_redis_operations.py", line 1479, in list_keys
azure.core.exceptions.HttpResponseError: Operation returned an invalid status 'Bad Request'
Content: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

cli.azure.cli.core.azclierror: Operation returned an invalid status 'Bad Request'
az_command_data_logger: Operation returned an invalid status 'Bad Request'
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x04017F10>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 1.936 seconds (init: 1.195, invoke: 0.741)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3251 in cache
telemetry.check: Negative: The C:\Users\bob\.azure\telemetry.txt was modified at 2023-03-03 09:59:01.242364, which in less than 600.000000 s

what the hell is that File "D:\a_work\1\s\build_scripts\windows\xxxxx ..... ??

ezYakaEagle442 commented 1 year ago

see

ezYakaEagle442 commented 1 year ago

@ezYakaEagle442 I was unable to reproduce this error. Could you run the command with the --debug flag and share the output?

Can you please try not on Windows, on a GitHub Workflow and VSCode/Terminal/WSL2/Ubuntu2204 ? (as there is no file path D:\a_work\1\s\build_scripts\windows\xxxxx .....)

ezYakaEagle442 commented 1 year ago

@PramodValavala-MSFT I share with you the Bicep template so that yo ucan test : 

@description('A UNIQUE name')
@maxLength(33)
param appName string = 'reddog${uniqueString(resourceGroup().id, subscription().id)}'

param location string = resourceGroup().location

param redisName string = 'redis-${appName}-poc' // redis-<project, app or service>-<environment>

// https://learn.microsoft.com/en-us/azure/templates/microsoft.cache/redis?pivots=deployment-language-bicep#sku
@allowed([
  'Basic'
  'Standard'
  'Premium'
]
)
param redisSkuName string = 'Premium'

@allowed([
  'C'
  'P'
]
)
param redisSkuFamily string = 'P'

resource redis 'Microsoft.Cache/redis@2022-06-01' = {
  name: redisName
  location: location
  properties: {
    sku: {
      name: redisSkuName
      family: redisSkuFamily
      capacity: 2
    }
    minimumTlsVersion: '1.2'
    publicNetworkAccess: 'Enabled'
    enableNonSslPort: false
  }
}

output redisResourceId string = redis.id
output redisName string = redis.name
output redisHost string = redis.properties.hostName
output redisPort int = redis.properties.port
output redisVersion string = redis.properties.redisVersion
//output redisPassword string = listKeys(redis.id, redis.apiVersion).primaryKey
ezYakaEagle442 commented 1 year ago

In the debuf trace, I see Request URL: 'https://management.azure.com/subscriptions/212121212121/resourceGroups/rg-aca-reddog-quarkus/providers/Microso ft.Cache/redis/redis-reddog7wwufb7rkg2oe-poc%0D/listKeys?api-version=2022-06-01'

The 0D suggest a CR in Hexa, same straneg thing at https://github.com/Azure/azure-cli/issues/25686

ezYakaEagle442 commented 1 year ago

I have tested and this issue does not happen running CLI on Windows :

FOR /F %i IN ('az deployment group show --name redis  -g rg-aca-reddog-quarkus --query properties.outputs.redisName.value -o tsv') DO set redis_name=%i

az redis list-keys -n %redis_name% -g rg-aca-reddog-quarkus

So this there is definetly a bug in CLI when running on Linux, it add a Carriage Return '0D' when running 'az deployment group show'

ezYakaEagle442 commented 1 year ago

it seems the issue is there around https://github.com/Azure/azure-cli/blob/azure-cli-2.45.0/src/azure-cli/azure/cli/command_modules/resource/custom.py#L411

ezYakaEagle442 commented 1 year ago

trim '\r' is a workaround : redis_name=$(az deployment group show --name redis -g rg-aca-reddog-quarkus --query properties.outputs.redisName.value -o tsv | tr -d '\r')

PramodValavala-MSFT commented 1 year ago

@ezYakaEagle442 Since the problem seems to be in the output of the az deployment group show command, I am closing this issue as a duplicate of #25647