search

Azure / azure-cli

Azure Command-Line Interface
MIT License
4.02k stars 2.99k forks source link

Unable to add azure-devops extension in Azure CLI v2.56.0 on Windows: ImportError: DLL load failed while importing _ctypes: The specified module could not be found. #28303

Open MattNipper opened 9 months ago

MattNipper commented 9 months ago

Describe the bug

With the current version of Azure CLI, I am unable to install the azure-devops extension on a Windows machine. This seems related to the python/pip packaged in CLI2.

Related command

az extension add --name bastion --debug

Errors

An error occurred. Pip failed with status code 1. Use --debug for more information.

Issue script & Debug output

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip__main.py", line 22, in from pip._internal.cli.main import main as _main File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\cli\main.py", line 10, in from pip._internal.cli.autocompletion import autocomplete File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\cli\autocompletion.py", line 10, in from pip._internal.cli.main_parser import create_main_parser File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\cli\main_parser.py", line 9, in from pip._internal.build_env import get_runnable_pip File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\build_env.py", line 19, in from pip._internal.cli.spinners import open_spinner File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\cli\spinners.py", line 9, in from pip._internal.utils.logging import get_indentation File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\utils\logging.py", line 29, in from pip._internal.utils.misc import ensure_dir File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\utils\misc.py", line 44, in from pip._internal.locations import get_major_minor_version File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\locations__init.py", line 14, in from . import _sysconfig File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\locations_sysconfig.py", line 11, in from .base import change_root, get_major_minor_version, is_osx_framework File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\locations\base.py", line 9, in from pip._internal.utils import appdirs File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_internal\utils\appdirs.py", line 13, in from pip._vendor import platformdirs as _appdirs File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_vendor\platformdirs__init__.py", line 46, in PlatformDirs = _set_platform_dir_class() #: Currently active platform ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_vendor\platformdirs\init__.py", line 26, in _set_platform_dir_class from pip._vendor.platformdirs.windows import Windows as Result File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip_vendor\platformdirs\windows.py", line 4, in import ctypes File "ctypes\init__.py", line 8, in ImportError: DLL load failed while importing _ctypes: The specified module could not be found.

Expected behavior

The extension should be installed.

Environment Summary

azure-cli 2.56.0

core 2.56.0 telemetry 1.1.0

Extensions: azure-batch-cli-extensions 7.0.0 bastion 0.2.5

Dependencies: msal 1.24.0b2 azure-mgmt-resource 23.1.0b2

Additional context

No response

azure-client-tools-bot-prd[bot] commented 9 months ago
Hi @MattNipper Find similar issue https://github.com/Azure/azure-cli/issues/23015.
Issue title azure-cli-core: Test failures with conflicting subparsers in Python 3.11.0b3
Create time 2022-06-24
Comment number 10

Please confirm if this resolves your issue.

yonzhan commented 9 months ago

Thank you for opening this issue, we will look into it.

jiasli commented 8 months ago

I can't repo this issue on my machine as both ctypes and _ctypes can be imported correctly.

> & "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe" -I -c "import ctypes; print(ctypes.__file__)"
C:\Program Files\Microsoft SDKs\Azure\CLI2\python311.zip\ctypes\__init__.pyc

> & "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe" -I -c "import _ctypes; print(_ctypes.__file__)"
C:\Program Files\Microsoft SDKs\Azure\CLI2\_ctypes.pyd

Can you verify if C:\Program Files\Microsoft SDKs\Azure\CLI2\_ctypes.pyd exists?

> Get-Item "C:\Program Files\Microsoft SDKs\Azure\CLI2\_ctypes.pyd"

    Directory: C:\Program Files\Microsoft SDKs\Azure\CLI2

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           12/4/2023  7:44 PM         124696 _ctypes.pyd

If not, I doubt the MSI installation is corrupted. Please try to perform a clean installation following https://github.com/Azure/azure-cli/issues/20990 and see if the error is gone.

wpdonders commented 8 months ago

I also face this issue (although on Azure CLI v2.58.0), and can confirm _ctypes.pyd exists on my machine. A clean installation does not fix the issue.

jiasli commented 8 months ago

@wpdonders, what is the output if you run the above commands?

BouvetMorten commented 8 months ago

I'm getting the same issue with the ML extension. Tried both version 2.58.0 and 2.57.0 of the CLI. Clean installation does not help.

Python version: 3.11.8 (Had Python 3.10 first, but tried upgrading to 3.11 unsuccessfully to solve the issue)

az version:

{
  "azure-cli": "2.57.0",
  "azure-cli-core": "2.57.0",
  "azure-cli-telemetry": "1.1.0",
  "extensions": {
    "bastion": "0.2.5",
    "containerapp": "0.3.46",
    "ssh": "2.0.2"
  }
}

Outputs from the above commands:

$ & "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe" -I -c "import _ctypes; print(_ctypes.__file__)"
C:\Program Files\Microsoft SDKs\Azure\CLI2\_ctypes.pyd
$ & "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe" -I -c "import ctypes; print(ctypes.__file__)"
C:\Program Files\Microsoft SDKs\Azure\CLI2\python311.zip\ctypes\__init__.pyc
$  Get-Item "C:\Program Files\Microsoft SDKs\Azure\CLI2\_ctypes.pyd"

    Directory: C:\Program Files\Microsoft SDKs\Azure\CLI2

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---          04/12/2023    19.44         124696 _ctypes.pyd

Output when running in --debug mode:

$ az extension add -n ml --debug
cli.knack.cli: Command arguments: ['extension', 'add', '-n', 'ml', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x000002A8014CB880>, <function OutputProducer.on_global_arguments at 0x000002A801656020>, <function CLIQuery.on_global_arguments at 0x000002A801683BA0>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'extension': ['azure.cli.command_modules.extension']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: extension                 0.003         1         7
cli.azure.cli.core: Total (1)                 0.003         1         7
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: Total (0)                 0.000         0         0
cli.azure.cli.core: Loaded 1 groups, 7 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : extension add
cli.azure.cli.core: Command table: extension add
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x000002A8044AAA20>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\morten.aursland\.azure\commands\2024-03-14.13-44-02.extension_add.17216.log'.
az_command_data_logger: command args: extension add -n {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x000002A8044E6E80>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x000002A80450CF40>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x000002A80450D080>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x000002A8016560C0>, <function CLIQuery.handle_query_parameter at 0x000002A801683C40>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x000002A80450CFE0>]
cli.azure.cli.core.extension.operations: Default enabled including preview versions for extension installation now. Disabled in May 2024. Use '--allow-preview true' to enable it specifically if needed. Use '--allow-preview false' to install stable version only.
urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 3787694
cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.10.0-py3-none-any.whl', 'ml-2.11.0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.12.1-py3-none-any.whl', 'ml-2.13.0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.14.1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.10.0-py3-none-any.whl', 'ml-2.11.0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.12.1-py3-none-any.whl', 'ml-2.13.0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.14.1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.10.0-py3-none-any.whl', 'ml-2.11.0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.12.1-py3-none-any.whl', 'ml-2.13.0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.14.1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['ml-2.0.0a1-py3-none-any.whl', 'ml-2.0.1a1-py3-none-any.whl', 'ml-2.0.1a2-py3-none-any.whl', 'ml-2.0.1a3-py3-none-any.whl', 'ml-2.0.1a4-py3-none-any.whl', 'ml-2.0.1a5-py3-none-any.whl', 'ml-2.0.2-py3-none-any.whl', 'ml-2.0.3-py3-none-any.whl', 'ml-2.0.7-py3-none-any.whl', 'ml-2.1.1-py3-none-any.whl', 'ml-2.1.2-py3-none-any.whl', 'ml-2.2.1-py3-none-any.whl', 'ml-2.2.2-py3-none-any.whl', 'ml-2.2.3-py3-none-any.whl', 'ml-2.3.1-py3-none-any.whl', 'ml-2.4.0-py3-none-any.whl', 'ml-2.4.1-py3-none-any.whl', 'ml-2.5.0-py3-none-any.whl', 'ml-2.6.0-py3-none-any.whl', 'ml-2.6.1-py3-none-any.whl', 'ml-2.7.0-py3-none-any.whl', 'ml-2.7.1-py3-none-any.whl', 'ml-2.8.0-py3-none-any.whl', 'ml-2.9.0-py3-none-any.whl', 'ml-2.10.0-py3-none-any.whl', 'ml-2.11.0-py3-none-any.whl', 'ml-2.12.0-py3-none-any.whl', 'ml-2.12.1-py3-none-any.whl', 'ml-2.13.0-py3-none-any.whl', 'ml-2.14.0-py3-none-any.whl', 'ml-2.14.1-py3-none-any.whl', 'ml-2.15.0-py3-none-any.whl', 'ml-2.15.1-py3-none-any.whl', 'ml-2.16.0-py3-none-any.whl', 'ml-2.17.0-py3-none-any.whl', 'ml-2.17.1-py3-none-any.whl', 'ml-2.17.2-py3-none-any.whl', 'ml-2.18.0-py3-none-any.whl', 'ml-2.19.0-py3-none-any.whl', 'ml-2.19.1-py3-none-any.whl', 'ml-2.20.0-py3-none-any.whl', 'ml-2.21.0-py3-none-any.whl', 'ml-2.21.1-py3-none-any.whl', 'ml-2.22.0-py3-none-any.whl', 'ml-2.23.0-py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Chosen {'downloadUrl': 'https://azuremlsdktestpypi.blob.core.windows.net/wheels/sdk-cli-v2-public/ml-2.23.0-py3-none-any.whl', 'filename': 'ml-2.23.0-py3-none-any.whl', 'metadata': {'azext.minCliCoreVersion': '2.15.0', 'classifiers': ['Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Environment :: Console', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: 3.9', 'Programming Language :: Python :: 3.10', 'License :: OSI Approved :: MIT License'], 'description_content_type': 'text/x-rst', 'extensions': {'python.details': {'contacts': [{'email': 'azuremlsdk@microsoft.com', 'name': 'Microsoft Corporation', 'role': 'author'}], 'document_names': {'description': 'DESCRIPTION.rst'}, 'project_urls': {'Home': 'https://github.com/Azure/azureml-examples'}}}, 'extras': [], 'generator': 'bdist_wheel (0.30.0)', 'license': 'MIT', 'metadata_version': '2.0', 'name': 'ml', 'run_requires': [{'requires': ['azure-common (<2.0.0,>=1.1)', 'azure-common<2.0.0,>=1.1', 'azure-mgmt-resource (<23.0.0,>=3.0.0)', 'azure-mgmt-resource<23.0.0,>=3.0.0', 'azure-mgmt-resourcegraph (<9.0.0,>=2.0.0)', 'azure-mgmt-resourcegraph<9.0.0,>=2.0.0', 'azure-storage-blob (<13.0.0,>=12.10.0)', 'azure-storage-blob<13.0.0,>=12.10.0', 'azure-storage-file-datalake (<13.0.0)', 'azure-storage-file-datalake<13.0.0', 'azure-storage-file-share (<13.0.0)', 'azure-storage-file-share<13.0.0', 'colorama (<0.5.0)', 'colorama<0.5.0', 'cryptography', 'cryptography', 'docker', 'docker', 'isodate', 'isodate', 'jsonschema (<5.0.0,>=4.0.0)', 'jsonschema<5.0.0,>=4.0.0', 'marshmallow (<4.0.0,>=3.5)', 'marshmallow<4.0.0,>=3.5', 'opencensus-ext-azure (<2.0.0)', 'opencensus-ext-azure<2.0.0', 'pydash (<7.0.6,>=6.0.0)', 'pydash<7.0.6,>=6.0.0', 'pyjwt (<3.0.0)', 'pyjwt<3.0.0', 'strictyaml (<2.0.0)', 'strictyaml<2.0.0', 'tqdm (<5.0.0)', 'tqdm<5.0.0', 'typing-extensions (<5.0.0)', 'typing-extensions<5.0.0']}], 'summary': 'Microsoft Azure Command-Line Tools AzureMachineLearningWorkspaces Extension', 'version': '2.23.0'}, 'sha256Digest': 'c4cffb5dd82c8146d11c93330b6952933bbe6755cdf77e954cdf910717e65323'}
cli.azure.cli.core.extension.operations: Extension source is url? True
cli.azure.cli.core.extension.operations: Downloading https://azuremlsdktestpypi.blob.core.windows.net/wheels/sdk-cli-v2-public/ml-2.23.0-py3-none-any.whl to C:\Users\MORTEN~1.AUR\AppData\Local\Temp\tmpu3djdfvi\ml-2.23.0-py3-none-any.whl
urllib3.connectionpool: Starting new HTTPS connection (1): azuremlsdktestpypi.blob.core.windows.net:443
urllib3.connectionpool: https://azuremlsdktestpypi.blob.core.windows.net:443 "GET /wheels/sdk-cli-v2-public/ml-2.23.0-py3-none-any.whl HTTP/1.1" 200 9334606
cli.azure.cli.core.extension.operations: Downloaded to C:\Users\MORTEN~1.AUR\AppData\Local\Temp\tmpu3djdfvi\ml-2.23.0-py3-none-any.whl
cli.azure.cli.core.extension.operations: Validating the extension C:\Users\MORTEN~1.AUR\AppData\Local\Temp\tmpu3djdfvi\ml-2.23.0-py3-none-any.whl
cli.azure.cli.core.extension.operations: Checksum of C:\Users\MORTEN~1.AUR\AppData\Local\Temp\tmpu3djdfvi\ml-2.23.0-py3-none-any.whl is OK
cli.azure.cli.core.extension.operations: Validation successful on C:\Users\MORTEN~1.AUR\AppData\Local\Temp\tmpu3djdfvi\ml-2.23.0-py3-none-any.whl
cli.azure.cli.core.extension.operations: Executing pip with args: ['install', '--target', 'C:\\Users\\morten.aursland\\.azure\\cliextensions\\ml', 'C:\\Users\\MORTEN~1.AUR\\AppData\\Local\\Temp\\tmpu3djdfvi\\ml-2.23.0-py3-none-any.whl']
cli.azure.cli.core.extension.operations: Running: ['C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'pip', 'install', '--target', 'C:\\Users\\morten.aursland\\.azure\\cliextensions\\ml', 'C:\\Users\\MORTEN~1.AUR\\AppData\\Local\\Temp\\tmpu3djdfvi\\ml-2.23.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']
cli.azure.cli.core.extension.operations: Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\__main__.py", line 22, in <module>
    from pip._internal.cli.main import main as _main
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\build_env.py", line 19, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\utils\logging.py", line 29, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\utils\misc.py", line 44, in <module>
    from pip._internal.locations import get_major_minor_version
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\locations\__init__.py", line 14, in <module>
    from . import _sysconfig
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\locations\_sysconfig.py", line 11, in <module>
    from .base import change_root, get_major_minor_version, is_osx_framework
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\locations\base.py", line 9, in <module>
    from pip._internal.utils import appdirs
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\utils\appdirs.py", line 13, in <module>
    from pip._vendor import platformdirs as _appdirs
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_vendor\platformdirs\__init__.py", line 46, in <module>
    PlatformDirs = _set_platform_dir_class()  #: Currently active platform
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_vendor\platformdirs\__init__.py", line 26, in _set_platform_dir_class
    from pip._vendor.platformdirs.windows import Windows as Result
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_vendor\platformdirs\windows.py", line 4, in <module>
    import ctypes
  File "ctypes\__init__.py", line 8, in <module>
ImportError: DLL load failed while importing _ctypes: The specified module could not be found.

cli.azure.cli.core.extension.operations: Command '['C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'pip', 'install', '--target', 'C:\\Users\\morten.aursland\\.azure\\cliextensions\\ml', 'C:\\Users\\MORTEN~1.AUR\\AppData\\Local\\Temp\\tmpu3djdfvi\\ml-2.23.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at C:\Users\morten.aursland\.azure\cliextensions\ml
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 664, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 729, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 698, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 334, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 349, in add_extension
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x000002A8044AACA0>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 12.044 seconds (init: 0.361, invoke: 11.684)
cli.azure.cli.core.decorators: Suppress exception:
Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 62, in <module>
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 55, in <module>
SystemExit: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/decorators.py", line 79, in _wrapped_func
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/telemetry.py", line 532, in _get_secrets_warning_config
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/config.py", line 147, in getboolean
ValueError: Not a boolean: None

telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3534 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\azure\cli\telemetry\__init__.pyc C:\Users\morten.aursland\.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.
jenshnielsen commented 8 months ago

I ran into a similar issue when installing the devops extension. I was able to workaround it by explicitly adding the path to the azure CLIs libs dir (the one containing _ctypes.pyd) to python path using the PYTHONPATH env variable before installing the extension.

In powershell

$ENV:PYTHONPATH = "C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2"

Install extension

Close console after this so modification to python path are not accidentally used for other python work.

MattNipper commented 8 months ago

I ran into a similar issue when installing the devops extension. I was able to workaround it by explicitly adding the path to the azure CLIs libs dir (the one containing _ctypes.pyd) to python path using the PYTHONPATH env variable before installing the extension.

In powershell

$ENV:PYTHONPATH = "C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2"

Install extension

Close console after this so modification to python path are not accidentally used for other python work.

Thanks @jenshnielsen! This also worked for me.

BouvetMorten commented 8 months ago

I ran into a similar issue when installing the devops extension. I was able to workaround it by explicitly adding the path to the azure CLIs libs dir (the one containing _ctypes.pyd) to python path using the PYTHONPATH env variable before installing the extension.

In powershell

$ENV:PYTHONPATH = "C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2"

Install extension

Close console after this so modification to python path are not accidentally used for other python work.

Thank you! It worked for me as well!

jenshnielsen commented 7 months ago

@jiasli for completeness my work around above should make it clear that the issue here is not corrupt or missing libraries but the fact that when python is loaded from within the azure cli extension manager "C:\Program Files\Microsoft SDKs\Azure\CLI2" is not correctly on path and libraries in that dir may therefore not be loaded correctly.

Scar11 commented 7 months ago

I ran into a similar issue when installing the devops extension. I was able to workaround it by explicitly adding the path to the azure CLIs libs dir (the one containing _ctypes.pyd) to python path using the PYTHONPATH env variable before installing the extension. In powershell

$ENV:PYTHONPATH = "C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2"

Install extension Close console after this so modification to python path are not accidentally used for other python work.

Thanks @jenshnielsen! This also worked for me.

You're a genius. Worked for me as well. Thanks @jenshnielsen! !

ant0nsc commented 7 months ago

@jiasli @yonzhan FYI, I was facing the same problem when trying to install the bastion extension. I was able to work around it by using an older CLI version, 2.51.0 - Thanks to @pj250183 for suggesting that here https://github.com/Azure/azure-cli-extensions/issues/7244

bebound commented 7 months ago

Although I can't repro this in my machine, the issue might stem from the different directory structure between embedded Python and standard Python.

In embedding Python, _ctypes is in \_ctypes.pyd. In standard Python, it's in \DLLs\_ctypes.pyd.

Here is a similar issue that pip fails to install package in embedded Python because of built-in module path: https://github.com/python/cpython/issues/100399 https://github.com/pypa/pip/pull/11724

Unfortunately, running pip install in embedded python is not officially supported by CPython or pip. We can't get help from them.

Using pip to manage dependencies as for a regular Python installation is not supported with this distribution, though with some care it may be possible to include and use pip for automatic updates. -- https://docs.python.org/3/using/windows.html#the-embeddable-package

We may move pyd and dll files to DLLs during package build, but I'm not certain if this is the correct approach

natke commented 3 months ago

I ran into a similar issue when installing the devops extension. I was able to workaround it by explicitly adding the path to the azure CLIs libs dir (the one containing _ctypes.pyd) to python path using the PYTHONPATH env variable before installing the extension.

In powershell

$ENV:PYTHONPATH = "C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2"

Install extension

Close console after this so modification to python path are not accidentally used for other python work.

This workaround did not work for me:

python --version
Python 3.11.9
az --version
azure-cli                         2.62.0

core                              2.62.0
telemetry                          1.1.0

Dependencies:
msal                              1.28.1
azure-mgmt-resource               23.1.1

Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\nakersha\.azure\cliextensions'

Python (Windows) 3.11.8 (tags/v3.11.8:db85d51, Feb  6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.
(azureml) C:\Users\nakersha\Develop\samples\llama>SET
PYTHONPATH="C:\Program Files\Microsoft SDKs\Azure\CLI2"
az extension add -n ml --debug

cli.azure.cli.core.extension.operations: Running: ['C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'pip', 'install', '--target', 'C:\\Users\\nakersha\\.azure\\cliextensions\\ml', 'C:\\Users\\nakersha\\AppData\\Local\\Temp\\tmplqhidek3\\ml-2.28.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']
cli.azure.cli.core.extension.operations: Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\__main__.py", line 22, in <module>
    from pip._internal.cli.main import main as _main
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\main.py", line 11, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\build_env.py", line 19, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\utils\logging.py", line 4, in <module>
    import logging.handlers
  File "logging\handlers.py", line 26, in <module>
  File "socket.py", line 51, in <module>

    ^
ImportError: DLL load failed while importing _socket: The specified module could not be found.

cli.azure.cli.core.extension.operations: Command '['C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'pip', 'install', '--target', 'C:\\Users\\nakersha\\.azure\\cliextensions\\ml', 'C:\\Users\\nakersha\\AppData\\Local\\Temp\\tmplqhidek3\\ml-2.28.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at C:\Users\nakersha\.azure\cliextensions\ml
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 664, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 701, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 334, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 349, in add_extension
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
jenshnielsen commented 3 months ago

@natke I would suggest checking if the _socket.dll file is in the directory that you have added to the python path. If not you should look if it is in some other dir of the installation and add it there