search

Azure / azure-cli

Azure Command-Line Interface
MIT License
4.01k stars 2.98k forks source link

az cli jmesquery support for to_lower() / lower()? #29876

Open mcdonamw opened 1 month ago

mcdonamw commented 1 month ago

Describe the bug

I'm new to az cli so I don't know if this is an issue or a limitation/by design, but I'm trying to do a case-insensitive query and my research states I would use to_lower() (though examples I see say to use lower() instead), however if I try any of these, az cli complains that neither of those are valid functions.

Related command

az disk list

Errors

az disk list --query "[?contains(lower(name), 'myvalue')]" --output table
Invalid jmespath query supplied for `--query`: Unknown function: lower()

az disk list --query "[?contains(to_lower(name), 'myvalue')]" --output table
Invalid jmespath query supplied for `--query`: Unknown function: to_lower()

Issue script & Debug output

PS C:\Users\**********> az disk list --query "[?contains(lower(name), 'myvalue')]" --output table --debug
cli.knack.cli: Command arguments: ['disk', 'list', '--query', "[?contains(lower(name), 'myvalue')]", '--output', 'table', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
Enable VT mode.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x000002C6F685F880>, <function OutputProducer.on_global_arguments at 0x000002C6F69E60C0>, <function CLIQuery.on_global_arguments at 0x000002C6F6A13C40>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'disk': ['azure.cli.command_modules.vm']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: vm                        0.196        58       269
cli.azure.cli.core: Total (1)                 0.196        58       269
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: Total (0)                 0.000         0         0
cli.azure.cli.core: Loaded 58 groups, 269 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : disk list
cli.azure.cli.core: Command table: disk list
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x000002C6F987E340>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\**********\.azure\commands\2024-09-09.18-02-48.disk_list.14256.log'.
az_command_data_logger: command args: disk list --query {} --output {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x000002C6F98B67A0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x000002C6F98D87C0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x000002C6F98D8900>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x000002C6F69E6160>, <function CLIQuery.handle_query_parameter at 0x000002C6F6A13CE0>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x000002C6F98D8860>]
cli.azure.cli.core.auth.persistence: build_persistence: location='C:\\Users\\**********\\.azure\\msal_token_cache.bin', encrypt=True
cli.azure.cli.core.auth.binary_cache: load: C:\Users\**********\.azure\msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: Initializing with Entra authority: https://login.microsoftonline.com/*******
msal.authority: openid_config("https://login.microsoftonline.com/*******/v2.0/.well-known/openid-configuration") = {'token_endpoint': 'https://login.microsoftonline.com/*******/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/*******/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/*******/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/*******/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/*******/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/*******/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/*******/kerberos', 'tenant_region_scope': 'NA', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'}
msal.application: Broker enabled? True
cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), claims=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: *******
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/*******/providers/Microsoft.Compute/disks?api-version=2023-04-02'
cli.azure.cli.core.sdk.policies: Request method: 'GET'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies:     'Accept': 'application/json'
cli.azure.cli.core.sdk.policies:     'x-ms-client-request-id': '*******'
cli.azure.cli.core.sdk.policies:     'CommandName': 'disk list'
cli.azure.cli.core.sdk.policies:     'ParameterSetName': '--query --output --debug'
cli.azure.cli.core.sdk.policies:     'User-Agent': 'AZURECLI/2.61.0 (MSI) azsdk-python-core/1.28.0 Python/3.11.8 (Windows-10-10.0.22631-SP0)'
cli.azure.cli.core.sdk.policies:     'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: This request has no body
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "GET /subscriptions/*******/providers/Microsoft.Compute/disks?api-version=2023-04-02 HTTP/1.1" 200 313648
cli.azure.cli.core.sdk.policies: Response status: 200
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies:     'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Content-Length': '313648'
cli.azure.cli.core.sdk.policies:     'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies:     'Expires': '-1'
cli.azure.cli.core.sdk.policies:     'x-ms-original-request-ids': '*******, *******'
cli.azure.cli.core.sdk.policies:     'x-ms-ratelimit-remaining-subscription-reads': '499'
cli.azure.cli.core.sdk.policies:     'x-ms-ratelimit-remaining-subscription-global-reads': '7499'
cli.azure.cli.core.sdk.policies:     'x-ms-request-id': '*******'
cli.azure.cli.core.sdk.policies:     'x-ms-correlation-request-id': '*******'
cli.azure.cli.core.sdk.policies:     'x-ms-routing-request-id': '*******'
cli.azure.cli.core.sdk.policies:     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.sdk.policies:     'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies:     'X-Cache': 'CONFIG_NOCACHE'
cli.azure.cli.core.sdk.policies:     'X-MSEdge-Ref': 'Ref A: ********** Ref B: ********** Ref C: 2024-09-09T22:02:49Z'
cli.azure.cli.core.sdk.policies:     'Date': 'Mon, 09 Sep 2024 22:02:49 GMT'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"value":[{"name":**********}
cli.knack.cli: Event: CommandInvoker.OnTransformResult [<function _resource_group_transform at 0x000002C6F98B59E0>, <function _x509_from_base64_to_hex_transform at 0x000002C6F98B5A80>]
cli.knack.cli: Event: CommandInvoker.OnFilterResult [<function CLIQuery.handle_query_parameter.<locals>.filter_output at 0x000002C6FA57CA40>]
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/functions.py", line 74, in call_function
KeyError: 'lower'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 676, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 170, in raise_event
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/query.py", line 45, in filter_output
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/parser.py", line 509, in search
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/visitor.py", line 94, in visit
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/visitor.py", line 180, in visit_filter_projection
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/visitor.py", line 94, in visit
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/visitor.py", line 169, in visit_function_expression
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/visitor.py", line 94, in visit
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/visitor.py", line 171, in visit_function_expression
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\jmespath/functions.py", line 76, in call_function
jmespath.exceptions.UnknownFunctionError: Unknown function: lower()

cli.azure.cli.core.azclierror: Invalid jmespath query supplied for `--query`: Unknown function: lower()
az_command_data_logger: Invalid jmespath query supplied for `--query`: Unknown function: lower()
To learn more about --query, please visit: 'https://docs.microsoft.com/cli/azure/query-azure-cli'
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x000002C6F987E5C0>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 2.148 seconds (init: 0.334, invoke: 1.815)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3802 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\azure\cli\telemetry\__init__.pyc C:\Users\**********\.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.

Expected behavior

Ability to search regardless of case

Environment Summary

Using Azure CloudShell

me [ ~ ]$ az --version
azure-cli                         2.64.0

core                              2.64.0
telemetry                          1.1.0

Extensions:
ai-examples                        0.2.5
ml                                2.29.0
ssh                                2.0.5
vm-repair                          1.0.7

Dependencies:
msal                              1.30.0
azure-mgmt-resource               23.1.1

Python location '/usr/bin/python3.9'
Extensions directory '/home/**********/.azure/cliextensions'
Extensions system directory '/usr/lib/python3.9/site-packages/azure-cli-extensions'

Python (Linux) 3.9.19 (main, Aug 23 2024, 00:07:48) 
[GCC 11.2.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response

yonzhan commented 1 month ago

Thank you for opening this issue, we will look into it.

danmacode commented 1 week ago

@mcdonamw Did you find another way to implement this functionality using the Azure CLI?