Open daniel-kajita opened 4 months ago
Thank you for opening this issue, we will look into it.
Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @dyu1208, @FumingZhang, @andyliuliming.
Issue related to extension k8s-runtime
. Could you please help take a look? +@ddadaal, @upxinxin
hi Daniel,
Have you tried reinstalling the extension? It seems that the azure.mgmt.kubernetesconfiguration
was dropped during the installation. Manually running pip install won't work because Azure CLI places dependencies in a different path.
hi Daniel,
Have you tried reinstalling the extension? It seems that the
azure.mgmt.kubernetesconfiguration
was dropped during the installation. Manually running pip install won't work because Azure CLI places dependencies in a different path.
Hello @HE-Xinyu,
I tried to uninstall and install it back, still have the same problem ! Adding the debug logs for removing and adding it back in an attached file, just in case if can help in your investigation.
az_extension_add_debug.txt az_extension_remove_debug.txt
Error persisted with :
Unable to load extension 'k8s-runtime: No module named 'azure.mgmt.kubernetesconfiguration''. Use --debug for more information.
'k8s-runtime' is misspelled or not recognized by the system.
Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs
@daniel-kajita The quantum issue has appeared before, but was closed without being resolved. https://github.com/Azure/azure-cli/issues/27452
I believe this is caused by the extension itself.
For dependencies starting with azure-xxxx
, the extension should vendor the dependencies instead of adding them into dependency list.
https://github.com/Azure/azure-cli-extensions/blob/60c9eb0eb7aab91bfa6a227fc1bf4d60e5033aaf/src/k8s-runtime/setup.py#L30-L31
The same goes for quantum extension https://github.com/Azure/azure-cli-extensions/blob/main/src/quantum/setup.py#L37 Hi @daniel-kajita You can ignore these two extensions until they vendored azure-related packages to their code and remove dependencies in setup.py.
@HE-Xinyu @warren-jones Please vendor(copy) azure-related packages to your extensions. Otherwise, these extensions may not be available and online documentation may not be automatically updated.
Describe the bug
I am trying to do a quick set-up of Azure Arc with a kind Kubernetes and a personal Azure Arc. I am able to connect fine with it but at the time to follow this guide it is failing with :
cli.azure.cli.core: Unable to load extension 'k8s-runtime: No module named 'azure.mgmt.kubernetesconfiguration''. Use --debug for more information
I tried to make sure I installed the Python dependencies like specified here.
pip3 install azure-mgmt-kubernetesconfiguration
Requirement already satisfied: azure-mgmt-kubernetesconfiguration in /usr/local/lib/python3.11/site-packages (3.1.0) Requirement already satisfied: isodate<1.0.0,>=0.6.1 in /usr/local/lib/python3.11/site-packages (from azure-mgmt-kubernetesconfiguration) (0.6.1) Requirement already satisfied: azure-common~=1.1 in /usr/local/lib/python3.11/site-packages (from azure-mgmt-kubernetesconfiguration) (1.1.28) Requirement already satisfied: azure-mgmt-core<2.0.0,>=1.3.2 in /usr/local/lib/python3.11/site-packages (from azure-mgmt-kubernetesconfiguration) (1.4.0) Requirement already satisfied: azure-core<2.0.0,>=1.26.2 in /usr/local/lib/python3.11/site-packages (from azure-mgmt-core<2.0.0,>=1.3.2->azure-mgmt-kubernetesconfiguration) (1.30.2) Requirement already satisfied: six in /usr/local/lib/python3.11/site-packages (from isodate<1.0.0,>=0.6.1->azure-mgmt-kubernetesconfiguration) (1.16.0) Requirement already satisfied: requests>=2.21.0 in /usr/local/lib/python3.11/site-packages (from azure-core<2.0.0,>=1.26.2->azure-mgmt-core<2.0.0,>=1.3.2->azure-mgmt-kubernetesconfiguration) (2.32.3) Requirement already satisfied: typing-extensions>=4.6.0 in /usr/local/lib/python3.11/site-packages (from azure-core<2.0.0,>=1.26.2->azure-mgmt-core<2.0.0,>=1.3.2->azure-mgmt-kubernetesconfiguration) (4.12.2) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/site-packages (from requests>=2.21.0->azure-core<2.0.0,>=1.26.2->azure-mgmt-core<2.0.0,>=1.3.2->azure-mgmt-kubernetesconfiguration) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/site-packages (from requests>=2.21.0->azure-core<2.0.0,>=1.26.2->azure-mgmt-core<2.0.0,>=1.3.2->azure-mgmt-kubernetesconfiguration) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/site-packages (from requests>=2.21.0->azure-core<2.0.0,>=1.26.2->azure-mgmt-core<2.0.0,>=1.3.2->azure-mgmt-kubernetesconfiguration) (2.2.1) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/site-packages (from requests>=2.21.0->azure-core<2.0.0,>=1.26.2->azure-mgmt-core<2.0.0,>=1.3.2->azure-mgmt-kubernetesconfiguration) (2024.6.2)
Related command
az k8s-runtime --debug load-balancer enable --resource-uri subscriptions/XXXXXXXX/resourceGroups/YYYYYY/providers/Microsoft.Kubernetes/connectedClusters/ZZZZZZZZZ
Errors
az k8s-runtime --debug load-balancer enable --resource-uri subscriptions/XXXXXXXX/resourceGroups/YYYYYY/providers/Microsoft.Kubernetes/connectedClusters/ZZZZZZZZZ Unable to load extension 'k8s-runtime: No module named 'azure.mgmt.kubernetesconfiguration''. Use --debug for more information. 'k8s-runtime' is misspelled or not recognized by the system.
Examples from AI knowledge base: https://aka.ms/cli_ref Read more about the command in reference docs
Issue script & Debug output
az k8s-runtime --debug load-balancer enable --resource-uri subscriptions/XXXXXXXX/resourceGroups/YYYYYY/providers/Microsoft.Kubernetes/connectedClusters/ZZZZZZZZZ cli.knack.cli: Command arguments: ['k8s-runtime', '--debug', 'load-balancer', 'enable', '--resource-uri', 'subscriptions/XXXXXXXX/resourceGroups/YYYYYY/providers/Microsoft.Kubernetes/connectedClusters/ZZZZZZZZZ'] cli.knack.cli: init debug log: Enable color in terminal. cli.knack.cli: Event: Cli.PreExecute [] cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x1102da200>, <function OutputProducer.on_global_arguments at 0x110921580>, <function CLIQuery.on_global_arguments at 0x11094f240>] cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate [] cli.azure.cli.core: No module found from index for '['k8s-runtime', '--debug', 'load-balancer', 'enable', '--resource-uri', 'subscriptions/XXXXXXXX/resourceGroups/YYYYYY/providers/Microsoft.Kubernetes/connectedClusters/ZZZZZZZZZ']' cli.azure.cli.core: Loading all modules and extensions cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'compute_recommender', 'config', 'configure', 'consumption', 'container', 'containerapp', 'cosmosdb', 'databoxedge', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'mysql', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm'] cli.azure.cli.core: Loading command modules: cli.azure.cli.core: Name Load Time Groups Commands cli.azure.cli.core: acr 0.119 36 149 cli.azure.cli.core: acs 0.027 14 76 cli.azure.cli.core: advisor 0.002 3 6 cli.azure.cli.core: ams 0.006 22 100 cli.azure.cli.core: apim 0.006 14 69 cli.azure.cli.core: appconfig 0.003 9 47 cli.azure.cli.core: appservice 0.053 79 270 cli.azure.cli.core: aro 0.009 1 10 cli.azure.cli.core: backup 0.003 16 60 cli.azure.cli.core: batch 0.023 34 102 cli.azure.cli.core: batchai 0.003 10 30 cli.azure.cli.core: billing 0.013 20 53 cli.azure.cli.core: botservice 0.003 12 42 cli.azure.cli.core: cdn 0.110 8 49 cli.azure.cli.core: cloud 0.002 1 7 cli.azure.cli.core: cognitiveservices 0.002 10 33 cli.azure.cli.core: compute_recommender 0.003 1 1 cli.azure.cli.core: config 0.001 2 7 cli.azure.cli.core: configure 0.001 2 5 cli.azure.cli.core: consumption 0.015 8 9 cli.azure.cli.core: container 0.008 1 11 cli.azure.cli.core: containerapp 0.084 36 115 cli.azure.cli.core: cosmosdb 0.013 58 199 cli.azure.cli.core: databoxedge 0.008 5 28 cli.azure.cli.core: dla 0.003 23 62 cli.azure.cli.core: dls 0.003 7 41 cli.azure.cli.core: dms 0.002 3 22 cli.azure.cli.core: eventgrid 0.004 25 96 cli.azure.cli.core: eventhubs 0.011 13 19 cli.azure.cli.core: extension 0.001 1 7 cli.azure.cli.core: feedback 0.001 1 2 cli.azure.cli.core: find 0.001 1 1 cli.azure.cli.core: hdinsight 0.006 8 39 cli.azure.cli.core: identity 0.002 2 11 cli.azure.cli.core: interactive 0.000 1 1 cli.azure.cli.core: iot 0.115 19 82 cli.azure.cli.core: keyvault 0.008 20 113 cli.azure.cli.core: kusto 0.002 3 14 cli.azure.cli.core: lab 0.003 11 34 cli.azure.cli.core: managedservices 0.001 3 8 cli.azure.cli.core: maps 0.001 5 13 cli.azure.cli.core: marketplaceordering 0.002 1 2 cli.azure.cli.core: monitor 0.426 17 61 cli.azure.cli.core: mysql 0.086 15 51 cli.azure.cli.core: netappfiles 0.038 8 17 cli.azure.cli.core: network 0.172 103 337 cli.azure.cli.core: policyinsights 0.016 9 17 cli.azure.cli.core: privatedns 0.023 14 60 cli.azure.cli.core: profile 0.002 2 8 cli.azure.cli.core: rdbms 0.021 49 202 cli.azure.cli.core: redis 0.003 7 38 cli.azure.cli.core: relay 0.022 7 8 cli.azure.cli.core: resource 0.011 51 231 cli.azure.cli.core: role 0.002 17 61 cli.azure.cli.core: search 0.007 7 19 cli.azure.cli.core: security 0.009 48 98 cli.azure.cli.core: servicebus 0.008 12 14 cli.azure.cli.core: serviceconnector 0.066 20 307 cli.azure.cli.core: servicefabric 0.015 27 80 cli.azure.cli.core: signalr 0.002 9 34 cli.azure.cli.core: sql 0.014 56 215 cli.azure.cli.core: sqlvm 0.024 4 20 cli.azure.cli.core: storage 0.039 59 273 cli.azure.cli.core: synapse 0.011 54 246 cli.azure.cli.core: util 0.001 3 7 cli.azure.cli.core: vm 0.052 58 233 cli.azure.cli.core: Total (66) 1.756 1205 4682 cli.azure.cli.core: Loading extensions: cli.azure.cli.core: Name Load Time Groups Commands Directory cli.azure.cli.core: connectedk8s 0.024 1 10 /Users/daniel/.azure/cliextensions/connectedk8s cli.azure.cli.core: customlocation 0.010 1 7 /Users/daniel/.azure/cliextensions/customlocation cli.azure.cli.core: k8s-extension 0.009 2 16 /Users/daniel/.azure/cliextensions/k8s-extension cli.knack.cli: Event: MainLoader.OnFailedExtensionLoad [] cli.azure.cli.core: Unable to load extension 'k8s-runtime: No module named 'azure.mgmt.kubernetesconfiguration''. Use --debug for more information. cli.azure.cli.core: Traceback (most recent call last): File "/usr/local/Cellar/azure-cli/2.61.0/libexec/lib/python3.11/site-packages/azure/cli/core/init.py", line 351, in _update_command_table_from_extensions _load_extension_command_loader(self, args, ext_mod) File "/usr/local/Cellar/azure-cli/2.61.0/libexec/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 1120, in _load_extension_command_loader return _load_command_loader(loader, args, ext, '') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Cellar/azure-cli/2.61.0/libexec/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 1106, in _load_command_loader command_table = command_loader.load_command_table(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/daniel/.azure/cliextensions/k8s-runtime/azext_k8s_runtime/init.py", line 22, in load_command_table from azext_k8s_runtime.commands import load_command_table File "/Users/daniel/.azure/cliextensions/k8s-runtime/azext_k8s_runtime/commands.py", line 13, in
from .custom import (enable_storage_class_cmd, disable_storage_class_cmd,
File "/Users/daniel/.azure/cliextensions/k8s-runtime/azext_k8s_runtime/custom.py", line 13, in
from .custom_commands.storage_class import enable_storage_class, disable_storage_class
File "/Users/daniel/.azure/cliextensions/k8s-runtime/azext_k8s_runtime/custom_commands/storage_class.py", line 18, in
from azure.mgmt.kubernetesconfiguration import SourceControlConfigurationClient
ModuleNotFoundError: No module named 'azure.mgmt.kubernetesconfiguration'
cli.azure.cli.core: Total (3) 0.042 4 33.add_subscription_parameter at 0x11169d6c0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x11169d760>, <function register_cache_arguments..add_cache_arguments at 0x11169d8a0>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x113035a80>]
cli.azure.cli.core.command_recommender: "--resource-uri" is an invalid parameter for command "k8s-runtime".
urllib3.connectionpool: Starting new HTTPS connection (1): app.aladdin.microsoft.com:443
urllib3.connectionpool: https://app.aladdin.microsoft.com:443 "GET /api/v1.0/suggestions?query=%7B%22command%22%3A+%22k8s-runtime%22%2C+%22parameters%22%3A+%22%22%7D&clientType=AzureCli&context=%7B%22versionNumber%22%3A+%222.61.0%22%2C+%22errorType%22%3A+%22UnknownSubcommand%22%2C+%22correlationId%22%3A+%224352a3f9-9a5e-40df-80c6-251aca3929f3%22%2C+%22subscriptionId%22%3A+%224c2596b7-866e-4990-ab6d-ce2fdacc783e%22%2C+%22eventId%22%3A+%226ee4ee7e-e580-4349-9e83-62114de7c80a%22%7D HTTP/1.1" 200 None
cli.azure.cli.core.command_recommender: "--resource-uri" is an invalid parameter for command "k8s-runtime".
cli.azure.cli.core.azclierror: NoneType: None
cli.azure.cli.core: Loaded 1195 groups, 4715 commands. cli.azure.cli.core: Updated command index in 0.004 seconds. cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x111634a40>] cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/Users/daniel/.azure/commands/2024-06-13.21-29-45.unknown_command.31099.log'. az_command_data_logger: command args: k8s-runtime --debug {} {} --resource-uri {} cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.
cli.azure.cli.core.azclierror: 'k8s-runtime' is misspelled or not recognized by the system. az_command_data_logger: 'k8s-runtime' is misspelled or not recognized by the system.
Examples from AI knowledge base: https://aka.ms/cli_ref Read more about the command in reference docs cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x111634cc0>] az_command_data_logger: exit code: 2 cli.main: Command ran in 2.413 seconds (init: 0.138, invoke: 2.275) telemetry.main: Begin splitting cli events and extra events, total events: 1 telemetry.client: Accumulated 0 events. Flush the clients. telemetry.main: Finish splitting cli events and extra events, cli events: 1 telemetry.save: Save telemetry record of length 3864 in cache telemetry.main: Begin creating telemetry upload process. telemetry.process: Creating upload process: "/usr/local/Cellar/azure-cli/2.61.0/libexec/bin/python /usr/local/Cellar/azure-cli/2.61.0/libexec/lib/python3.11/site-packages/azure/cli/telemetry/init.py /Users/daniel/.azure" telemetry.process: Return from creating process telemetry.main: Finish creating telemetry upload process.
Expected behavior
I would expect the Azure Arc LoadBalancer to be enabled
Environment Summary
az --version azure-cli 2.61.0
core 2.61.0 telemetry 1.1.0
Extensions: connectedk8s 1.7.3 customlocation 0.1.3 k8s-extension 1.6.1 k8s-runtime 1.0.1
Dependencies: msal 1.28.0 azure-mgmt-resource 23.1.1
Python location '/usr/local/Cellar/azure-cli/2.61.0/libexec/bin/python' Extensions directory '/Users/daniel/.azure/cliextensions'
Python (Darwin) 3.11.9 (main, Apr 2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
Additional context
No response