az ml extension shows as installed using official documentation but not working

egheorgheMSFT commented 6 months ago

Describe the bug

Following the official installation procedure on a fresh RHEL 8.9 VM, AzCLI doesn't seem to pick up on the ml extension after it being installed, instead producing this error:

The extension was installed according to the steps here:

[Install and set up the CLI (v2) - Azure Machine Learning | Microsoft Learn] (https://learn.microsoft.com/en-us/azure/machine-learning/how-to-configure-cli?view=azureml-api-2&tabs=public)

The extension shows as installed:

This was replicated on three different types of Azure Resources: a VMSS and two fresh VM on my side as well, for both RHEL 8.7 and 8.9.

Related command

az ml -h

Errors

No module named 'google.protobuf' 'ml' is mispelled or not recognized by the system

Issue script & Debug output

[root@testmlextension egheorghe]# az ml -h --debug cli.knack.cli: Command arguments: ['ml', '-h', '--debug'] cli.knack.cli: init debug log: Enable color in terminal. cli.knack.cli: Event: Cli.PreExecute [] cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f8b884c2160>, <function OutputProducer.on_global_arguments at 0x7f8b881d8d30>, <function CLIQuery.on_global_arguments at 0x7f8b87f6b1f0>] cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate [] cli.azure.cli.core: No module found from index for '['ml', '-h', '--debug']' cli.azure.cli.core: Loading all modules and extensions cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'config', 'configure', 'consumption', 'container', 'containerapp', 'cosmosdb', 'databoxedge', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'mysql', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm'] cli.azure.cli.core: Loading command modules: cli.azure.cli.core: Name Load Time Groups Commands cli.azure.cli.core: acr 0.111 36 149 cli.azure.cli.core: acs 0.015 14 76 cli.azure.cli.core: advisor 0.001 3 6 cli.azure.cli.core: ams 0.005 22 100 cli.azure.cli.core: apim 0.005 14 69 cli.azure.cli.core: appconfig 0.003 9 47 cli.azure.cli.core: appservice 0.061 79 270 cli.azure.cli.core: aro 0.007 1 10 cli.azure.cli.core: backup 0.003 16 60 cli.azure.cli.core: batch 0.023 34 102 cli.azure.cli.core: batchai 0.002 10 30 cli.azure.cli.core: billing 0.009 19 53 cli.azure.cli.core: botservice 0.003 12 42 cli.azure.cli.core: cdn 0.095 11 48 cli.azure.cli.core: cloud 0.001 1 7 cli.azure.cli.core: cognitiveservices 0.002 10 33 cli.azure.cli.core: config 0.001 2 7 cli.azure.cli.core: configure 0.001 2 5 cli.azure.cli.core: consumption 0.012 8 9 cli.azure.cli.core: container 0.006 1 11 cli.azure.cli.core: containerapp 0.069 36 115 cli.azure.cli.core: cosmosdb 0.012 58 199 cli.azure.cli.core: databoxedge 0.005 5 28 cli.azure.cli.core: dla 0.003 23 62 cli.azure.cli.core: dls 0.004 7 41 cli.azure.cli.core: dms 0.002 3 22 cli.azure.cli.core: eventgrid 0.004 25 96 cli.azure.cli.core: eventhubs 0.009 13 19 cli.azure.cli.core: extension 0.001 1 7 cli.azure.cli.core: feedback 0.001 1 2 cli.azure.cli.core: find 0.001 1 1 cli.azure.cli.core: hdinsight 0.004 8 39 cli.azure.cli.core: identity 0.001 2 11 cli.azure.cli.core: interactive 0.000 1 1 cli.azure.cli.core: iot 0.113 19 82 cli.azure.cli.core: keyvault 0.008 20 113 cli.azure.cli.core: kusto 0.002 3 14 cli.azure.cli.core: lab 0.003 11 34 cli.azure.cli.core: managedservices 0.001 3 8 cli.azure.cli.core: maps 0.001 5 13 cli.azure.cli.core: marketplaceordering 0.001 1 2 cli.azure.cli.core: monitor 0.377 18 61 cli.azure.cli.core: mysql 0.076 15 51 cli.azure.cli.core: netappfiles 0.029 8 16 cli.azure.cli.core: network 0.096 103 338 cli.azure.cli.core: policyinsights 0.008 9 17 cli.azure.cli.core: privatedns 0.014 14 60 cli.azure.cli.core: profile 0.001 2 8 cli.azure.cli.core: rdbms 0.016 49 202 cli.azure.cli.core: redis 0.002 7 38 cli.azure.cli.core: relay 0.016 7 8 cli.azure.cli.core: resource 0.099 51 228 cli.azure.cli.core: role 0.003 17 61 cli.azure.cli.core: search 0.006 7 19 cli.azure.cli.core: security 0.009 50 105 cli.azure.cli.core: servicebus 0.006 12 15 cli.azure.cli.core: serviceconnector 0.018 16 244 cli.azure.cli.core: servicefabric 0.009 27 76 cli.azure.cli.core: signalr 0.002 9 34 cli.azure.cli.core: sql 0.013 56 215 cli.azure.cli.core: sqlvm 0.015 4 20 cli.azure.cli.core: storage 0.047 59 273 cli.azure.cli.core: synapse 0.011 54 246 cli.azure.cli.core: util 0.001 3 7 cli.azure.cli.core: vm 0.044 58 233 cli.azure.cli.core: Total (65) 1.527 1205 4618 cli.azure.cli.core: Loaded 1192 groups, 4618 commands. cli.azure.cli.core: Updated command index in 0.005 seconds. cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f8b848bab80>] cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2024-04-15.07-18-02.unknown_command.9429.log'. az_command_data_logger: command args: ml -h --debug cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7f8b84860790>] cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad [] cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x7f8b84888820>, <function register_cache_arguments..add_cache_arguments at 0x7f8b84828670>] cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded [] cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7f8b81f5f0d0>] urllib3.connectionpool: Starting new HTTPS connection (1): app.aladdin.microsoft.com:443 urllib3.connectionpool: https://app.aladdin.microsoft.com:443 "GET /api/v1.0/suggestions?query=%7B%22command%22%3A+%22ml%22%2C+%22parameters%22%3A+%22--help%22%7D&clientType=AzureCli&context=%7B%22versionNumber%22%3A+%222.59.0%22%2C+%22errorType%22%3A+%22UnknownSubcommand%22%2C+%22correlationId%22%3A+%2239141af4-3e76-4fd0-a27b-de895d8d2382%22%2C+%22eventId%22%3A+%225c839d14-5917-4a1a-a626-6c50b8ad4d91%22%7D HTTP/1.1" 200 None cli.azure.cli.core.command_recommender: "--name" is an invalid parameter for command "ml". cli.azure.cli.core.azclierror: NoneType: None

cli.azure.cli.core.azclierror: 'ml' is misspelled or not recognized by the system. az_command_data_logger: 'ml' is misspelled or not recognized by the system.

Expected behavior

az ml -h shows the available commands

Environment Summary

[root@testmlextension egheorghe]# az --version azure-cli 2.59.0

core 2.59.0 telemetry 1.1.0

Dependencies: msal 1.27.0 azure-mgmt-resource 23.1.0b2

Python location '/bin/python3.9' Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response

yonzhan commented 6 months ago

Thank you for opening this issue, we will look into it.

microsoft-github-policy-service[bot] commented 6 months ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

egheorgheMSFT commented 6 months ago

@yonzhan Hello Yong, were you able to review this by any chance? Many thanks :)

needuv commented 6 months ago

@egheorgheMSFT , there are some known issues with installing the mlextension on RHEL. Please try the following steps as a workaround to get unblocked:

Manually install Python 3.11 : sudo dnf install python311
Install azure-cli through pip: python3.11 -m pip install azure-cli
Use the pip-installed azure-cli to install and use the ml extension: (The command may look something like: ~/.local/bin/az extension add --name ml )

egheorgheMSFT commented 6 months ago

@needuv thank you, the customer has just confirmed the workaround works for now

Azure / azure-cli-extensions