az cli throws error on az login and other commands

dewa55 commented 3 years ago

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name az login

Errors:

The command failed with an unexpected error. Here is the traceback:
__init__() got an unexpected keyword argument 'user_agent'
Traceback (most recent call last):
  File "/home/vedran/.local/lib/python3.8/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/command_modules/profile/custom.py", line 152, in login
    subscriptions = profile.find_subscriptions_on_login(
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 201, in find_subscriptions_on_login
    subscriptions = subscription_finder.find_through_authorization_code_flow(
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 929, in find_through_authorization_code_flow
    result = self._find_using_common_tenant(token_entry[_ACCESS_TOKEN], resource)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 973, in _find_using_common_tenant
    client = self._arm_client_factory(token_credential)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 891, in create_arm_client_factory
    client = client_type(credentials, api_version=api_version,
TypeError: __init__() got an unexpected keyword argument 'user_agent'

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az login

Expected Behavior

Environment Summary

Linux-5.11.0-34-generic-x86_64-with-glibc2.29, Ubuntu 20.04.3 LTS
Python 3.8.10
Installer: PIP

azure-cli 2.28.0

Extensions:
azure-devops 0.17.0

Additional Context

yonzhan commented 3 years ago

@jiasli for awareness

jiasli commented 3 years ago

Could be the same as https://github.com/Azure/azure-cli/issues/18626

dewa55 commented 3 years ago

I have tried to reinstall azure-cli python3 -m pip install --upgrade --force-reinstall azure-cli but no luck. I started to get this error: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. launchpadlib 1.10.13 requires testresources, which is not installed. azure-storage-file 2.1.0 requires azure-storage-common~=2.1, but you have azure-storage-common 1.4.2 which is incompatible. Successfully installed MarkupSafe-2.0.1 PyGithub-1.55 PyJWT-2.1.0 PyNaCl-1.4.0 PySocks-1.7.1 adal-1.2.7 antlr4-python3-runtime-4.7.2 applicationinsights-0.11.10 argcomplete-1.12.3 azure-appconfiguration-1.1.1 azure-batch-11.0.0 azure-cli-2.28.1 azure-cli-core-2.28.1 azure-cli-telemetry-1.0.6 azure-common-1.1.27 azure-core-1.18.0 azure-cosmos-3.2.0 azure-datalake-store-0.0.52 azure-functions-devops-build-0.0.22 azure-graphrbac-0.60.0 azure-identity-1.6.1 azure-keyvault-1.1.0 azure-keyvault-administration-4.0.0b3 azure-loganalytics-0.1.0 azure-mgmt-advisor-9.0.0 azure-mgmt-apimanagement-0.2.0 azure-mgmt-appconfiguration-2.0.0 azure-mgmt-applicationinsights-1.0.0 azure-mgmt-authorization-0.61.0 azure-mgmt-batch-16.0.0 azure-mgmt-batchai-7.0.0b1 azure-mgmt-billing-6.0.0 azure-mgmt-botservice-0.3.0 azure-mgmt-cdn-11.0.0 azure-mgmt-cognitiveservices-12.0.0 azure-mgmt-compute-22.1.0 azure-mgmt-consumption-2.0.0 azure-mgmt-containerinstance-8.0.0 azure-mgmt-containerregistry-8.1.0 azure-mgmt-containerservice-16.1.0 azure-mgmt-core-1.2.2 azure-mgmt-cosmosdb-6.4.0 azure-mgmt-databoxedge-1.0.0 azure-mgmt-datalake-analytics-0.2.1 azure-mgmt-datalake-nspkg-3.0.1 azure-mgmt-datalake-store-0.5.0 azure-mgmt-datamigration-9.0.0 azure-mgmt-deploymentmanager-0.2.0 azure-mgmt-devtestlabs-4.0.0 azure-mgmt-dns-8.0.0 azure-mgmt-eventgrid-9.0.0 azure-mgmt-eventhub-4.1.0 azure-mgmt-extendedlocation-1.0.0b2 azure-mgmt-hdinsight-8.0.0 azure-mgmt-imagebuilder-0.4.0 azure-mgmt-iotcentral-9.0.0b1 azure-mgmt-iothub-2.1.0 azure-mgmt-iothubprovisioningservices-0.3.0 azure-mgmt-keyvault-9.1.0 azure-mgmt-kusto-0.3.0 azure-mgmt-loganalytics-11.0.0 azure-mgmt-managedservices-1.0.0 azure-mgmt-managementgroups-0.2.0 azure-mgmt-maps-2.0.0 azure-mgmt-marketplaceordering-1.1.0 azure-mgmt-media-7.0.0 azure-mgmt-monitor-2.0.0 azure-mgmt-msi-0.2.0 azure-mgmt-netapp-4.0.0 azure-mgmt-network-19.0.0 azure-mgmt-nspkg-3.0.2 azure-mgmt-policyinsights-1.0.0 azure-mgmt-privatedns-1.0.0 azure-mgmt-rdbms-9.1.0 azure-mgmt-recoveryservices-2.0.0 azure-mgmt-recoveryservicesbackup-0.14.0 azure-mgmt-redhatopenshift-1.0.0 azure-mgmt-redis-13.0.0 azure-mgmt-relay-0.1.0 azure-mgmt-reservations-0.6.0 azure-mgmt-resource-19.0.0 azure-mgmt-search-8.0.0 azure-mgmt-security-1.0.0 azure-mgmt-servicebus-6.0.0 azure-mgmt-servicefabric-1.0.0 azure-mgmt-servicefabricmanagedclusters-1.0.0 azure-mgmt-signalr-1.0.0b2 azure-mgmt-sql-3.0.1 azure-mgmt-sqlvirtualmachine-1.0.0b1 azure-mgmt-storage-18.0.0 azure-mgmt-synapse-2.0.0 azure-mgmt-trafficmanager-0.51.0 azure-mgmt-web-2.0.0 azure-multiapi-storage-0.6.2 azure-nspkg-3.0.2 azure-storage-common-1.4.2 azure-synapse-accesscontrol-0.5.0 azure-synapse-artifacts-0.6.0 azure-synapse-managedprivateendpoints-0.3.0 azure-synapse-spark-0.2.0 bcrypt-3.2.0 certifi-2021.5.30 cffi-1.14.6 chardet-4.0.0 colorama-0.4.4 cryptography-3.3.2 deprecated-1.2.13 distro-1.6.0 fabric-2.6.0 humanfriendly-9.2 idna-2.10 invoke-1.6.0 isodate-0.6.0 javaproperties-0.5.1 jinja2-3.0.1 jmespath-0.10.0 jsondiff-1.2.0 knack-0.8.2 msal-1.14.0 msal-extensions-0.3.0 msrest-0.6.21 msrestazure-0.6.4 oauthlib-3.1.1 packaging-20.9 paramiko-2.7.2 pathlib2-2.3.6 pkginfo-1.7.1 portalocker-1.7.1 psutil-5.8.0 pycparser-2.20 pygments-2.10.0 pyopenssl-20.0.1 pyparsing-2.4.7 python-dateutil-2.8.2 pytz-2019.1 pyyaml-5.4.1 requests-2.25.1 requests-oauthlib-1.3.0 scp-0.13.6 semver-2.13.0 six-1.16.0 sshtunnel-0.1.5 tabulate-0.8.9 urllib3-1.26.6 vsts-0.1.25 websocket-client-0.56.0 wrapt-1.12.1 xmltodict-0.12.0

After installing launchpadlib I am getting error: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. azure-storage-file 2.1.0 requires azure-storage-common~=2.1, but you have azure-storage-common 1.4.2 which is incompatible.

After trying to install that package it fails with another dependency and it goes on and on

jiasli commented 3 years ago

azure-cli requires https://github.com/Azure/azure-cli/blob/ea46ad4a424a63406f40f732a9d76258623fd12a/src/azure-cli/setup.py#L133
azure-storage-file 2.1.0 requires azure-storage-common~=2.1

Therefore, you can't install azure-cli with azure-storage-file in the same environment. We recommend installing Azure CLI following the official methods. If you have to use pip to install, you can install azure-cli in a separate virtual environment (https://github.com/Azure/azure-cli/issues/18626#issuecomment-887411446).

jiasli commented 3 years ago

Meanwhile, @evelyn-ys, perhaps azure-cli can bump the dependency? https://github.com/Azure/azure-cli/blob/ea46ad4a424a63406f40f732a9d76258623fd12a/src/azure-cli/setup.py#L133

evelyn-ys commented 3 years ago

Thanks @jiasli Storage module should all depend on azure-mgmt-storage and azure-multiapi-storage. I'll remove azure-storage-common after checking no dependency on that.

dewa55 commented 3 years ago

azure-cli requires https://github.com/Azure/azure-cli/blob/ea46ad4a424a63406f40f732a9d76258623fd12a/src/azure-cli/setup.py#L133

azure-storage-file 2.1.0 requires azure-storage-common~=2.1

Therefore, you can't install azure-cli with azure-storage-file in the same environment. We recommend installing Azure CLI following the official methods. If you have to use pip to install, you can install azure-cli in a separate virtual environment (#18626 (comment)).

I have installed azure-cli trough Ubuntu repo. I think problem started when I installed all modules in VS Code for Azure. I have tried reinstalling azure-cli with pip and apt.

jiasli commented 3 years ago

I think problem started when I installed all modules in VS Code for Azure.

I am not aware VS Code as a concept of "module". I am also not aware there is any VS Code extension that will install Azure CLI through pip. Could you share some reference?

Does apt installed Azure CLI work for you?

We don' recommend installing with pip in a shared environment because it can always cause unexcepted dependency conflicts, such as https://github.com/Azure/azure-cli/issues/18584.

dewa55 commented 3 years ago

I am not aware VS Code as a concept of "module". I am also not aware there is any VS Code extension that will install Azure CLI through pip. Could you share some reference? Sorry they are called extensions. I have installed this https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-node-azure-pack

Does apt installed Azure CLI work for you? No. I have reinstalled and still the same

jiasli commented 3 years ago

No. I have reinstalled and still the same

You may check which az is invoked with

command -v az

The one installed by apt is /usr/bin/az.
The one installed by pip is /usr/local/bin/az.

You may delete the one installed by pip with

pip uninstall azure-cli

dewa55 commented 3 years ago

Hello Jisali so this is what I did

/home/vedran/.local/bin/az

$pip uninstall azure-cli
Found existing installation: azure-cli 2.28.1
Uninstalling azure-cli-2.28.1:
  Would remove:
    /home/vedran/.local/bin/az
    /home/vedran/.local/bin/az.bat
    /home/vedran/.local/bin/az.completion.sh
    /home/vedran/.local/lib/python3.8/site-packages/azure/cli/__main__.py
    /home/vedran/.local/lib/python3.8/site-packages/azure/cli/command_modules/*
    /home/vedran/.local/lib/python3.8/site-packages/azure_cli-2.28.1.dist-info/*
Proceed (Y/n)? Y
  Successfully uninstalled azure-cli-2.28.1

$ sudo apt install azure-cli
Reading package lists... Done
Building dependency tree       
Reading state information... Done
azure-cli is already the newest version (2.0.81+ds-4ubuntu0.2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

$az login
The default web browser has been opened at https://login.microsoftonline.com/common/oauth2/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.
You have logged in. Now let us find all the subscriptions to which you have access...
The command failed with an unexpected error. Here is the traceback:
__init__() got an unexpected keyword argument 'user_agent'
Traceback (most recent call last):
  File "/home/vedran/.local/lib/python3.8/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/usr/lib/python3/dist-packages/azure/cli/command_modules/profile/custom.py", line 142, in login
    subscriptions = profile.find_subscriptions_on_login(
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 201, in find_subscriptions_on_login
    subscriptions = subscription_finder.find_through_authorization_code_flow(
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 929, in find_through_authorization_code_flow
    result = self._find_using_common_tenant(token_entry[_ACCESS_TOKEN], resource)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 973, in _find_using_common_tenant
    client = self._arm_client_factory(token_credential)
  File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 891, in create_arm_client_factory
    client = client_type(credentials, api_version=api_version,
TypeError: __init__() got an unexpected keyword argument 'user_agent'
To open an issue, please run: 'az feedback'

$ command -v az
/usr/local/bin/az

jiasli commented 3 years ago

This 2.0.81 is not the Azure CLI distributed by Microsoft. You need to follow https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt to install it from Microsoft Linux repo.

The dependency chain seems to be messed up:

File "/usr/lib/python3/dist-packages/azure/cli/command_modules/profile/custom.py", line 142, in login
    subscriptions = profile.find_subscriptions_on_login(
File "/home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py", line 201, in find_subscriptions_on_login
    subscriptions = subscription_finder.find_through_authorization_code_flow(

The Azure CLI installed under /usr/lib/python3/dist-packages/azure/cli/command_modules/profile/custom.py (installed by apt) calls /home/vedran/.local/lib/python3.8/site-packages/azure/cli/core/_profile.py (installed by pip).

Is it possible to remove everything under /home/vedran/.local to start from scratch? Please make sure you have nothing important in it.

rm -r /home/vedran/.local

dewa55 commented 3 years ago

My steps were:

sudo apt purge azure-cli
sudo rm -r /home/vedran/.local
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

ran az login

Error loading command module 'acr'
Error loading command module 'ams'
Error loading command module 'apim'
Error loading command module 'appconfig'
Error loading command module 'appservice'
Error loading command module 'backup'
Error loading command module 'botservice'
Error loading command module 'cdn'
Error loading command module 'configure'
Error loading command module 'consumption'
Error loading command module 'cosmosdb'
Error loading command module 'deploymentmanager'
Error loading command module 'dla'
Error loading command module 'dls'
Error loading command module 'iot'
Error loading command module 'lab'
Error loading command module 'monitor'
Error loading command module 'netappfiles'
Error loading command module 'network'
Error loading command module 'reservations'
Error loading command module 'search'
Error loading command module 'security'
Error loading command module 'servicefabric'
Error loading command module 'sqlvm'
Error loading command module 'storage'
Error loading command module 'vm'
Unable to load extension 'azure-devops'. Use --debug for more information.
You have logged in. Now let us find all the subscriptions to which you have access...

but the login was successful and I see everything. I think this is now from vs code and his extensions. I will reinstall the extension and start again.

EDIT: Unistall and Install of azure extensions in VS Code did not help seams like this issue: https://github.com/Azure/azure-cli/issues/15517

dewa55 commented 3 years ago

I have fixed above with sudo pip freeze | grep azure | xargs sudo pip uninstall -y sudo rm -rf /usr/lib/python3/dist-packages/azure* After that in VS Code on Azure extension Uninstall and Install.

Thank you @jiasli for help

jiasli commented 3 years ago

@dewa55, I am glad to know it works for you.

This time the issue is not caused by pip, but apt which installs unofficial Azure CLI from Ubuntu repo, judging by the installation folder /usr/lib/python3/dist-packages/azure* (notice the dist-packages). This Azure CLI is not maintained by Microsoft (#19640).

The official Azure CLI is installed at /opt/az/lib/python3.6/site-packages/ (notice the site-packages).

Yes, sudo rm -rf /usr/lib/python3/dist-packages/azure* will do the job. An easier way is to run sudo apt remove azure-cli -y && sudo apt autoremove -y to remove this unofficial Azure CLI.

Azure / azure-cli