Open dem4gus opened 2 years ago
This error is a blocker to my workflow so I started to roll back through the versions of azure-cli
in the rpm repository to try to get it working. The command also failed in 2.35.0 but succeeded in 2.34.1. When I installed 2.34.1 dnf
(using the 1.el7 repository instead of 1.el8, since only 2.35 and 2.36 are available in 1.el8) also resolved python3.6 as a dependency, which means it got installed on my system. After python3.6 was installed I ran az upgrade
to return to Azure CLI 2.36.0, but the version of Python in use by Azure CLI is still 3.6.15. After returning to Azure CLI 2.36 with Python 3.6, the az storage
commands are functioning because Python3.6 is still available for Azure CLI to use.
To summarize:
libpython3.6m.so.1.0
listed as a dependency, which installs Python3.6:
# dnf repoquery --requires azure-cli-2.34.1-1.el7
Last metadata expiration check: 0:02:27 ago on Fri Apr 29 18:45:26 2022.
ld-linux-x86-64.so.2()(64bit)
ld-linux-x86-64.so.2(GLIBC_2.3)(64bit)
libc.so.6(GLIBC_2.14)(64bit)
libpthread.so.0()(64bit)
libpthread.so.0(GLIBC_2.2.5)(64bit)
libpython3.6m.so.1.0()(64bit)
python3
rtld(GNU_HASH
# dnf repoquery --requires azure-cli-2.35.0-1.el7
Last metadata expiration check: 0:02:41 ago on Fri Apr 29 18:45:26 2022.
ld-linux-x86-64.so.2()(64bit)
ld-linux-x86-64.so.2(GLIBC_2.3)(64bit)
libc.so.6(GLIBC_2.14)(64bit)
libpthread.so.0()(64bit)
libpthread.so.0(GLIBC_2.2.5)(64bit)
python3
rtld(GNU_HASH)
az upgrade
, Python3.6 is still available:
# az upgrade
...
# az -v
azure-cli 2.36.0
core 2.36.0 telemetry 1.0.6
Dependencies: msal 1.17.0 azure-mgmt-resource 20.0.0
Python location '/usr/bin/python3.6' Extensions directory '/root/.azure/cliextensions'
Python (Linux) 3.6.15 (default, Mar 3 2022, 00:00:00) [GCC 11.2.1 20220127 (Red Hat 11.2.1-9)]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
- `az storage` commands now succeed on Azure CLI 2.35+
Please run 'az login' to setup account.
To further confirm, removing python3.6 causes the commands to again fail:
...
azure-cli 2.36.0
core 2.36.0 telemetry 1.0.6
Dependencies: msal 1.17.0 azure-mgmt-resource 20.0.0
Python location '/usr/bin/python3' Extensions directory '/root/.azure/cliextensions'
Python (Linux) 3.10.2 (main, Jan 17 2022, 00:00:00) [GCC 11.2.1 20211203 (Red Hat 11.2.1-7)]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
The command failed with an unexpected error. Here is the traceback: 'NoneType' object is not iterable Traceback (most recent call last): File "/usr/lib64/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke cmd_result = self.invocation.execute(args) File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 556, in execute self.commands_loader.load_arguments(command) File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/init.py", line 516, in load_arguments loader.load_arguments(command) # this adds entries to the argument registries File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/init.py", line 31, in load_arguments load_arguments(self, command) File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/_params.py", line 683, in load_arguments get_permission_help_string(t_account_permissions)), File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/_validators.py", line 837, in get_permission_help_string return ' '.join(['({}){}'.format(x[0], x[1:]) for x in allowed_values]) TypeError: 'NoneType' object is not iterable To open an issue, please run: 'az feedback'
Storage
@dem4gus, it actually meets my expectation that the el8 package doesn't work for Fedora 35! 😂
We don't really have official support for fedora 35. The doc https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=dnf is somehow outdated.
Before we dive into this issue, you may install with pip
to get unblocked quickly: https://github.com/Azure/azure-cli/issues/20476
Please let us know if it works for you.
The problem is that why libpython3.6m.so.1.0()(64bit)
is missing from the RPM's requirement since 2.35.0. We didn't change anything in the build process. 🤔
There are the build logs for
I manually binary-searched all builds on dev
branch between 2.34.1 and 2.35.0, and saw libpython3.6m.so.1.0
was removed by 20220309.2 (triggered by commit 07819726e13d72587eb82b65c92ecba3bb009f32).
Requires: ld-linux-x86-64.so.2()(64bit) ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) libc.so.6()(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3)(64bit) libc.so.6(GLIBC_2.3.2)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libc.so.6(GLIBC_2.6)(64bit) libc.so.6(GLIBC_2.7)(64bit) libffi-c643fa1a.so.6.0.4()(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) rtld(GNU_HASH)
By comparing 20220309.2 with previous build 20220308.33, I noticed wrapt
was bumped:
- wrapt-1.11.2
+ wrapt-1.14.0
Then by backtracking PRs before 07819726e13d72587eb82b65c92ecba3bb009f32, I saw #21557 removed the limitation on wrapt
, which has dependency on libpython3.6m.so.1.0()(64bit)
. This can be verified with /usr/lib/rpm/find-requires
:
> docker run -it --rm centos:centos7
# yum install -y wget rpm-build gcc libffi-devel python3-devel openssl-devel make bash coreutils diffutils patch dos2unix python3-virtualenv
# python3 -m venv cli-env
# . cli-env/bin/activate
# pip install -U pip
# pip install wrapt==1.11.2
# cd cli-env
# find -name *.so
./lib/python3.6/site-packages/wrapt/_wrappers.cpython-36m-x86_64-linux-gnu.so
# find -name *.so | /usr/lib/rpm/find-requires
libc.so.6()(64bit)
libc.so.6(GLIBC_2.2.5)(64bit)
libc.so.6(GLIBC_2.4)(64bit)
libpthread.so.0()(64bit)
libpython3.6m.so.1.0()(64bit)
There is only python3
on CentOS7, but no python3.6
, so there is no simple way to pin Python to python3.6
for RPM built with CentOS 7. Actually, it is a bad practice to build RPM on one system (CentOS 7) and install/run it on another system (Fedora 35).
To completely solve this problem, we should use Fedora 35 to build an RPM and install that RPM on Fedora 35 (https://github.com/Azure/azure-cli/issues/22278).
Regarding /usr/lib/rpm/find-requires
:
An old similar issue:
A workaround (credit to @bebound. Thanks!) is to specify python36
here:
python36
is not a package name, but a capability provided by python3
or python 3.6
package.
On Fedora 35:
# dnf repoquery --provides python3.6
Last metadata expiration check: 0:11:48 ago on Fri May 6 09:11:57 2022.
bundled(libmpdec) = 2.4.2
bundled(mpdecimal) = 2.4.2
libpython3.6m.so.1.0
libpython3.6m.so.1.0()(64bit)
pkgconfig(python-3.6) = 3.6
pkgconfig(python-3.6m) = 3.6
python3.6 = 3.6.15-1.fc35
python3.6 = 3.6.15-2.fc35
python3.6(x86-32) = 3.6.15-1.fc35
python3.6(x86-32) = 3.6.15-2.fc35
python3.6(x86-64) = 3.6.15-1.fc35
python3.6(x86-64) = 3.6.15-2.fc35
python36 = 3.6.15-1.fc35
python36 = 3.6.15-2.fc35
On CentOS 7:
# yum -y install yum-utils
# repoquery --provides python3
python(abi) = 3.6
python3 = 3.6.8-18.el7
python3(x86-32) = 3.6.8-18.el7
python36 = 3.6.8-18.el7
python36(x86-32) = 3.6.8-18.el7
python(abi) = 3.6
python3 = 3.6.8-18.el7
python3(x86-64) = 3.6.8-18.el7
python36 = 3.6.8-18.el7
python36(x86-64) = 3.6.8-18.el7
Since Python 3.6 is now deprecated, the better solution is to build RPMs with the latest Fedora.
See
Debug log shows why some import failed:
# az storage account create -h --debug
...
cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/profiles/_shared.py", line 614, in _get_attr
op = import_module(full_mod_path)
File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/lib64/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_11_09/blob/__init__.py", line 6, in <module>
from .appendblobservice import AppendBlobService
File "/usr/lib64/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_11_09/blob/appendblobservice.py", line 30, in <module>
from ._deserialization import (
File "/usr/lib64/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_11_09/blob/_deserialization.py", line 41, in <module>
from ._encryption import _decrypt_blob
File "/usr/lib64/az/lib/python3.6/site-packages/azure/multiapi/storage/v2018_11_09/blob/_encryption.py", line 13, in <module>
from cryptography.hazmat.primitives.padding import PKCS7
File "/usr/lib64/az/lib/python3.6/site-packages/cryptography/hazmat/primitives/padding.py", line 13, in <module>
from cryptography.hazmat.bindings._padding import lib
ModuleNotFoundError: No module named '_cffi_backend'
cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/profiles/_shared.py", line 614, in _get_attr
op = import_module(full_mod_path)
File "/usr/lib64/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/lib64/az/lib/python3.6/site-packages/azure/multiapi/storagev2/fileshare/__init__.py", line 1, in <module>
__import__('pkg_resources').declare_namespace(__name__)
File "/usr/lib64/az/lib/python3.6/site-packages/pkg_resources/__init__.py", line 77, in <module>
__import__('pkg_resources.extern.packaging.requirements')
File "/usr/lib64/az/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 9, in <module>
from pkg_resources.extern.pyparsing import stringStart, stringEnd, originalTextFor, ParseException
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 672, in _load_unlocked
File "<frozen importlib._bootstrap>", line 632, in _load_backward_compatible
File "/usr/lib64/az/lib/python3.6/site-packages/pkg_resources/extern/__init__.py", line 43, in load_module
__import__(extant)
File "/usr/lib64/az/lib/python3.6/site-packages/pkg_resources/_vendor/pyparsing.py", line 943, in <module>
collections.MutableMapping.register(ParseResults)
AttributeError: module 'collections' has no attribute 'MutableMapping'
This is because Python 3.10 is used to import Python 3.6 libraries which are not compatible.
I hit similar error as well
az storage account list --debug
cli.knack.cli: Command arguments: ['storage', 'account', 'list', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f5a0fc3bc40>, <function OutputProducer.on_global_arguments at 0x7f5a0f998720>, <function CLIQuery.on_global_arguments at 0x7f5a0f9e23e0>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'storage': ['azure.cli.command_modules.storage']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: storage 0.050 58 272
cli.azure.cli.core: Total (1) 0.050 58 272
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: Total (0) 0.000 0 0
cli.azure.cli.core: Loaded 58 groups, 272 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : storage account list
cli.azure.cli.core: Command table: storage account list
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f5a0eb7ed40>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/usr/local/google/home/chunla/.azure/commands/2023-08-24.18-18-40.storage_account_list.1771692.log'.
az_command_data_logger: command args: storage account list --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.
cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/azure/cli/core/profiles/_shared.py", line 651, in _get_attr
op = import_module(full_mod_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "
cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/azure/cli/core/profiles/_shared.py", line 651, in _get_attr
op = import_module(full_mod_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "
cli.azure.cli.core.profiles._shared: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/azure/cli/core/profiles/_shared.py", line 655, in _get_attr op = getattr(op, part) ^^^^^^^^^^^^^^^^^ AttributeError: module 'azure.mgmt.storage.v2022_09_01.models' has no attribute 'ListKeyExpand'
cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/azure/cli/core/profiles/_shared.py", line 651, in _get_attr
op = import_module(full_mod_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "
cli.azure.cli.core.azclierror: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/core/commands/init.py", line 561, in execute self.commands_loader.load_arguments(command) File "/usr/lib/python3/dist-packages/azure/cli/core/init.py", line 522, in load_arguments loader.load_arguments(command) # this adds entries to the argument registries ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/init.py", line 31, in load_arguments load_arguments(self, command) File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/_params.py", line 750, in load_arguments get_permission_help_string(t_account_permissions)), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/_validators.py", line 870, in get_permission_help_string return ' '.join(['({}){}'.format(x[0], x[1:]) for x in allowed_values]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable
cli.azure.cli.core.azclierror: The command failed with an unexpected error. Here is the traceback: az_command_data_logger: The command failed with an unexpected error. Here is the traceback: cli.azure.cli.core.azclierror: 'NoneType' object is not iterable Traceback (most recent call last): File "/usr/lib/python3/dist-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/core/commands/init.py", line 561, in execute self.commands_loader.load_arguments(command) File "/usr/lib/python3/dist-packages/azure/cli/core/init.py", line 522, in load_arguments loader.load_arguments(command) # this adds entries to the argument registries ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/init.py", line 31, in load_arguments load_arguments(self, command) File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/_params.py", line 750, in load_arguments get_permission_help_string(t_account_permissions)), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/_validators.py", line 870, in get_permission_help_string return ' '.join(['({}){}'.format(x[0], x[1:]) for x in allowed_values]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable az_command_data_logger: 'NoneType' object is not iterable Traceback (most recent call last): File "/usr/lib/python3/dist-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/core/commands/init.py", line 561, in execute self.commands_loader.load_arguments(command) File "/usr/lib/python3/dist-packages/azure/cli/core/init.py", line 522, in load_arguments loader.load_arguments(command) # this adds entries to the argument registries ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/init.py", line 31, in load_arguments load_arguments(self, command) File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/_params.py", line 750, in load_arguments get_permission_help_string(t_account_permissions)), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/azure/cli/command_modules/storage/_validators.py", line 870, in get_permission_help_string return ' '.join(['({}){}'.format(x[0], x[1:]) for x in allowed_values]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable To check existing issues, please visit: https://github.com/Azure/azure-cli/issues cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f5a0eb7efc0>] az_command_data_logger: exit code: 1 cli.azure.cli.main: Command ran in 1.200 seconds (init: 0.258, invoke: 0.942) telemetry.main: Begin splitting cli events and extra events, total events: 1 telemetry.client: Accumulated 0 events. Flush the clients. telemetry.main: Finish splitting cli events and extra events, cli events: 1 telemetry.save: Save telemetry record of length 7159 in cache telemetry.check: Returns Positive. telemetry.main: Begin creating telemetry upload process. telemetry.process: Creating upload process: "/usr/bin/python3 /usr/lib/python3/dist-packages/azure/cli/telemetry/init.py /usr/local/google/home/chunla/.azure" telemetry.process: Return from creating process telemetry.main: Finish creating telemetry upload process.
Describe the bug
Any
az storage
commands fail on Fedora 35Command Name
az storage account list
Errors:
To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.
docker run -it index.docker.io/library/fedora:35
rpm --import https://packages.microsoft.com/keys/microsoft.asc
dnf install -y https://packages.microsoft.com/config/rhel/8/packages-microsoft-prod.rpm
dnf install azure-cli
az login --use-device-code
az storage account list
Expected Behavior
Available storage accounts in the subscription will be listed, or command will fail gracefully.
Environment Summary
Additional Context
The same command with the same account succeeds when run from a container using
mcr.microsoft.com/azure-cli
.