Our Flux extension was in Creating state all day yesterday. Today, we noticed it is in Failed state, but we cannot uninstall it. The error from Azure CLI we are getting:
Related command
az k8s-extension delete
Errors
The command failed with an unexpected error. Here is the traceback:
No module named 'azure.mgmt.relay'
Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 562, in execute
self.commands_loader.load_arguments(command)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/init.py", line 521, in load_arguments
self.command_table[command].load_arguments() # this loads the arguments via reflection
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 319, in load_arguments
super(AzCliCommand, self).load_arguments()
File "/usr/lib64/az/lib/python3.9/site-packages/knack/commands.py", line 104, in load_arguments
cmd_args = self.arguments_loader()
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 125, in arguments_loader
op = self.get_op_handler(self.op_path)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
handler = import_module(mod_to_import)
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home//.azure/cliextensions/k8s-extension/azext_k8s_extension/custom.py", line 29, in
from .partner_extensions.AzureMLKubernetes import AzureMLKubernetes
File "/home//.azure/cliextensions/k8s-extension/azext_k8s_extension/partner_extensions/AzureMLKubernetes.py", line 15, in
import azure.mgmt.relay
ModuleNotFoundError: No module named 'azure.mgmt.relay'
Issue script & Debug output
cli.knack.cli: Command arguments: ['k8s-extension', 'delete', '--resource-group', '', '--cluster-name', '', '--cluster-type', 'managedClusters', '--name', 'flux', '--yes', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7faa9b6e7160>, <function OutputProducer.on_global_arguments at 0x7faa9b601d30>, <function CLIQuery.on_global_arguments at 0x7faa9b598310>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'k8s-extension': ['azext_k8s_extension']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: Total (0) 0.000 0 0
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: ai-examples 0.247 1 1 /usr/lib/python3.9/site-packages/azure-cli-extensions/ai-examples
cli.azure.cli.core: k8s-extension 0.005 2 9 /home//.azure/cliextensions/k8s-extension
cli.azure.cli.core: Total (2) 0.252 3 10
cli.azure.cli.core: Loaded 3 groups, 10 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : k8s-extension delete
cli.azure.cli.core: Command table: k8s-extension delete
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7faa9aa47820>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home//.azure/commands/2024-08-02.07-51-07.k8s-extension_delete.1703.log'.
az_command_data_logger: command args: k8s-extension delete --resource-group {} --cluster-name {} --cluster-type {} --name {} --yes --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7faa9a9ef430>]
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 562, in execute
self.commands_loader.load_arguments(command)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/init.py", line 521, in load_arguments
self.command_table[command].load_arguments() # this loads the arguments via reflection
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 319, in load_arguments
super(AzCliCommand, self).load_arguments()
File "/usr/lib64/az/lib/python3.9/site-packages/knack/commands.py", line 104, in load_arguments
cmd_args = self.arguments_loader()
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 125, in arguments_loader
op = self.get_op_handler(self.op_path)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
handler = import_module(mod_to_import)
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home//azure/cliextensions/k8s-extension/azext_k8s_extension/custom.py", line 29, in
from .partner_extensions.AzureMLKubernetes import AzureMLKubernetes
File "/home//.azure/cliextensions/k8s-extension/azext_k8s_extension/partner_extensions/AzureMLKubernetes.py", line 15, in
import azure.mgmt.relay
ModuleNotFoundError: No module named 'azure.mgmt.relay'
Describe the bug
Our Flux extension was in Creating state all day yesterday. Today, we noticed it is in Failed state, but we cannot uninstall it. The error from Azure CLI we are getting:
Related command
az k8s-extension delete
Errors
The command failed with an unexpected error. Here is the traceback: No module named 'azure.mgmt.relay' Traceback (most recent call last): File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 562, in execute self.commands_loader.load_arguments(command) File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/init.py", line 521, in load_arguments self.command_table[command].load_arguments() # this loads the arguments via reflection File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 319, in load_arguments super(AzCliCommand, self).load_arguments() File "/usr/lib64/az/lib/python3.9/site-packages/knack/commands.py", line 104, in load_arguments cmd_args = self.arguments_loader() File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 125, in arguments_loader op = self.get_op_handler(self.op_path) File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 59, in get_op_handler handler = import_module(mod_to_import) File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home//.azure/cliextensions/k8s-extension/azext_k8s_extension/custom.py", line 29, in
from .partner_extensions.AzureMLKubernetes import AzureMLKubernetes
File "/home//.azure/cliextensions/k8s-extension/azext_k8s_extension/partner_extensions/AzureMLKubernetes.py", line 15, in
import azure.mgmt.relay
ModuleNotFoundError: No module named 'azure.mgmt.relay'
Issue script & Debug output
cli.knack.cli: Command arguments: ['k8s-extension', 'delete', '--resource-group', '', '--cluster-name', '', '--cluster-type', 'managedClusters', '--name', 'flux', '--yes', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7faa9b6e7160>, <function OutputProducer.on_global_arguments at 0x7faa9b601d30>, <function CLIQuery.on_global_arguments at 0x7faa9b598310>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'k8s-extension': ['azext_k8s_extension']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: Total (0) 0.000 0 0
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: ai-examples 0.247 1 1 /usr/lib/python3.9/site-packages/azure-cli-extensions/ai-examples
cli.azure.cli.core: k8s-extension 0.005 2 9 /home//.azure/cliextensions/k8s-extension
cli.azure.cli.core: Total (2) 0.252 3 10.add_subscription_parameter at 0x7faa9a9ef430>]
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 562, in execute
self.commands_loader.load_arguments(command)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/init.py", line 521, in load_arguments
self.command_table[command].load_arguments() # this loads the arguments via reflection
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 319, in load_arguments
super(AzCliCommand, self).load_arguments()
File "/usr/lib64/az/lib/python3.9/site-packages/knack/commands.py", line 104, in load_arguments
cmd_args = self.arguments_loader()
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 125, in arguments_loader
op = self.get_op_handler(self.op_path)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
handler = import_module(mod_to_import)
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home//azure/cliextensions/k8s-extension/azext_k8s_extension/custom.py", line 29, in
from .partner_extensions.AzureMLKubernetes import AzureMLKubernetes
File "/home//.azure/cliextensions/k8s-extension/azext_k8s_extension/partner_extensions/AzureMLKubernetes.py", line 15, in
import azure.mgmt.relay
ModuleNotFoundError: No module named 'azure.mgmt.relay'
cli.azure.cli.core: Loaded 3 groups, 10 commands. cli.azure.cli.core: Found a match in the command table. cli.azure.cli.core: Raw command : k8s-extension delete cli.azure.cli.core: Command table: k8s-extension delete cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7faa9aa47820>] cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home//.azure/commands/2024-08-02.07-51-07.k8s-extension_delete.1703.log'. az_command_data_logger: command args: k8s-extension delete --resource-group {} --cluster-name {} --cluster-type {} --name {} --yes --debug cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.
Expected behavior
The extension be deleted.
Environment Summary
azure-cli 2.62.0
core 2.62.0 telemetry 1.1.0
Extensions: ai-examples 0.2.5 aks-preview 0.5.128 azure-firewall 0.14.5 k8s-extension 1.4.0 ml 2.28.0 ssh 2.0.4
Dependencies: msal 1.28.1 azure-mgmt-resource 23.1.1
Python location '/usr/bin/python3.9' Extensions directory '/home//.azure/cliextensions'
Extensions system directory '/usr/lib/python3.9/site-packages/azure-cli-extensions'
Python (Linux) 3.9.19 (main, Mar 28 2024, 18:56:59) [GCC 11.2.0]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
Additional context
No response