in my company we have been facing issues with some scripts that uses az add extension command to install azure-devops:
it showed the already reported #17938 long standing issue
We included a bunch of workarounds & tests, seems that the only working approach was to disable AZURE_SSL_VERIFICATION (but this could trigger other issues).
However, today I realized that the issue was not all anything on the script, it's just the inconsistency of the own command:
Fresh powershell terminal
Run subsequent 10 times in a row the command (intervaled 5 secs - 45 secs depending if --debug option used& the installation is succesfull)
The results of the command is sometimes 3 without ssl verification issue, 7 with them; others 9 out of 10 fails, 8/10 works & so on. Tested in 3 different machines in our company.
For info, we have Zscaler in the company, but setting HTTP_PROXY & HTTPS_PROXY makes no difference.
Related command
az add extension --name azure-devops
Errors
Unable to get extension index.
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='aka.ms', port=443): Max retries exce
eded with url: /azure-cli-extension-index-v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATEVERIFY
FAILED] certificate verify failed: certificate signature failure (_ssl.c:1006)')))
Issue script & Debug output
Fail log:
az : DEBUG: cli.knack.cli: Command arguments: ['extension', 'add', '--name', 'azure-devops', '--debug']
At line:1 char:1
+ az extension add --name azure-devops --debug 2>&1 > .\add_extension2. ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (DEBUG: cli.knac...ps', '--debug']:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x00000
1DA67776FC0>, <function OutputProducer.on_global_arguments at 0x000001DA6789C900>, <function CLIQuery.on_global_argumen
ts at 0x000001DA678C6520>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'extension': ['azure.cli.command_modules.extension']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name Load Time Groups Commands
DEBUG: cli.azure.cli.core: extension 0.004 1 7
DEBUG: cli.azure.cli.core: Total (1) 0.004 1 7
DEBUG: cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
DEBUG: cli.azure.cli.core: Loading extensions:
DEBUG: cli.azure.cli.core: Name Load Time Groups Commands Directory
DEBUG: cli.azure.cli.core: Total (0) 0.000 0 0
DEBUG: cli.azure.cli.core: Loaded 1 groups, 7 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command : extension add
DEBUG: cli.azure.cli.core: Command table: extension add
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging
at 0x000001DA695C5440>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\UsersUser\.azure\commands\
2024-02-16.12-18-50.extension_add.24048.log'.
INFO: az_command_data_logger: command args: extension add --name {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>
.add_subscription_parameter at 0x000001DA695FCC20>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_
arguments at 0x000001DA695FF880>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x000001DA695FF9C0
>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x00000
1DA6789C9A0>, <function CLIQuery.handle_query_parameter at 0x000001DA678C65C0>, <function register_ids_argument.<locals
>.parse_ids_arguments at 0x000001DA695FF920>]
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 467
, in _make_request
self._validate_conn(conn)
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 109
6, in _validate_conn
conn.connect()
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 642, in
connect
sock_and_verified = _ssl_wrap_socket_and_match_hostname(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 782, in
_ssl_wrap_socket_and_match_hostname
ssl_sock = ssl_wrap_socket(
^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl_.py", line 470, in
ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl_.py", line 514, in
_ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 1108, in _create
self.do_handshake()
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 1379, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate signature failure
(_ssl.c:1006)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 790
, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 491
, in _make_request
raise new_e
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate signature failure
(_ssl.c:1006)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 486, in
send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 844
, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py", line 515, in
increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='aka.ms', port=443): Max retries exceeded with url: /azure-c
li-extension-index-v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate ver
ify failed: certificate signature failure (_ssl.c:1006)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\extension\_index.py",
line 47, in get_index
response = requests.get(index_url, verify=(not should_disable_connection_verify()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 73, in get
return request("get", url, params=params, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 59, in reques
t
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 589, in
request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 703, in
send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 517, in
send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='aka.ms', port=443): Max retries exceeded with url: /azure-cli-e
xtension-index-v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
failed: certificate signature failure (_ssl.c:1006)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\knack\cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\commands\__init__.py"
, line 663, in execute
raise ex
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\commands\__init__.py"
, line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\commands\__init__.py"
, line 697, in _run_job
result = cmd_copy(params)
^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\commands\__init__.py"
, line 333, in __call__
return self.handler(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\commands\command_oper
ation.py", line 121, in handler
return op(**command_args)
^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\command_modules\extension\
custom.py", line 16, in add_extension_cmd
return add_extension(cli_ctx=cmd.cli_ctx, source=source, extension_name=extension_name, index_url=index_url,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\extension\operations.
py", line 318, in add_extension
source, ext_sha256 = resolve_from_index(extension_name, index_url=index_url, target_version=version, cli_ctx=cmd_cl
i_ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\extension\_resolve.py
", line 100, in resolve_from_index
candidates = get_index_extensions(index_url=index_url, cli_ctx=cli_ctx).get(extension_name, [])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\extension\_index.py",
line 64, in get_index_extensions
index = get_index(index_url=index_url, cli_ctx=cli_ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\core\extension\_index.py",
line 57, in get_index
raise CLIError(msg)
knack.util.CLIError: Unable to get extension index.
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='aka.ms', port=443): Max retries exce
eded with url: /azure-cli-extension-index-v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_
FAILED] certificate verify failed: certificate signature failure (_ssl.c:1006)')))
ERROR: cli.azure.cli.core.azclierror: Unable to get extension index.
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='aka.ms', port=443): Max retries exce
eded with url: /azure-cli-extension-index-v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_
FAILED] certificate verify failed: certificate signature failure (_ssl.c:1006)')))
ERROR: az_command_data_logger: Unable to get extension index.
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='aka.ms', port=443): Max retries exce
eded with url: /azure-cli-extension-index-v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_
FAILED] certificate verify failed: certificate signature failure (_ssl.c:1006)')))
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x000001DA695C56C0>
]
INFO: az_command_data_logger: exit code: 1
INFO: cli.__main__: Command ran in 3.176 seconds (init: 1.130, invoke: 2.046)
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3737 in cache
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "C:\UsersUser\AppData\Local\Programs\Python\Python311\python.exe
C:\UsersUser\AppData\Local\Programs\Python\Python311\Lib\site-packages\azure\cli\telemetry\__init__.py C:\Users\SFD
XJAY\.azure"
INFO: telemetry.process: Return from creating process
INFO: telemetry.main: Finish creating telemetry upload process.
Describe the bug
Hello,
in my company we have been facing issues with some scripts that uses az add extension command to install azure-devops:
We included a bunch of workarounds & tests, seems that the only working approach was to disable AZURE_SSL_VERIFICATION (but this could trigger other issues).
However, today I realized that the issue was not all anything on the script, it's just the inconsistency of the own command:
Fresh powershell terminal
Run subsequent 10 times in a row the command (intervaled 5 secs - 45 secs depending if --debug option used& the installation is succesfull)
The results of the command is sometimes 3 without ssl verification issue, 7 with them; others 9 out of 10 fails, 8/10 works & so on. Tested in 3 different machines in our company.
For info, we have Zscaler in the company, but setting HTTP_PROXY & HTTPS_PROXY makes no difference.
Related command
az add extension --name azure-devops
Errors
Unable to get extension index. Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='aka.ms', port=443): Max retries exce eded with url: /azure-cli-extension-index-v1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATEVERIFY FAILED] certificate verify failed: certificate signature failure (_ssl.c:1006)')))
Issue script & Debug output
Fail log:
OK Log:
Differences of seconds between executions.
Expected behavior
az add extension --name azure-devops works consistently.
Environment Summary
Tested with azure-cli 2.54, 2.56 & 2.57 and different Python versions (3.10, 3.9, 3.11.5, 3.11.8)
azure-cli 2.57.0
core 2.57.0 telemetry 1.1.0
Extensions: azure-devops 0.26.0
Dependencies: msal 1.26.0 azure-mgmt-resource 23.1.0b2
Python location 'C:\Users\User\AppData\Local\Programs\Python\Python311\python.exe' Extensions directory 'C:\Users\User.azure\cliextensions'
Python (Windows) 3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:38:34) [MSC v.1936 64 bit (AMD64)]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
Additional context
No response