search

Azure / azure-cli

Azure Command-Line Interface
MIT License
3.93k stars 2.91k forks source link

Support Python 3.12 #27673

Open chenrui333 opened 9 months ago

chenrui333 commented 9 months ago

Description

Python 3.12 has been released on 2023-10-02: https://www.python.org/downloads/release/python-3120/

Azure CLI needs to support Python 3.12 like how we supported 3.10 (#19857), 3.11 (#24494).

TODOs

azure-client-tools-bot-prd[bot] commented 9 months ago
Hi @chenrui333 Find similar issue https://github.com/Azure/azure-cli/issues/19857.
Issue title Support Python 3.10
Create time 2021-10-12
Comment number 6

Possible solution: As we did for Python 3.10 and 3.11, Azure CLI needs to support Python 3.12. We can follow the same steps we did for Python 3.10 support, which includes updating dependencies like pytest, pylint, and ensuring all test cases pass. Additionally, we need to make sure that Knack and azdev also support Python 3.12. We can refer to the pull requests for Knack and azdev that added support for Python 3.10 and update them accordingly for Python 3.12. Once we have updated the dependencies and ensured that all test cases pass, we can release a new version of Azure CLI that supports Python 3.12.

Please confirm if this resolves your issue.

yonzhan commented 9 months ago

Thank you for opening this issue, we will look into it.

chenrui333 commented 9 months ago

I find there is no open issue to support py3.12, create this one for the team.

Gyarbij commented 9 months ago

Any progress on this and will this impact App Service Python 3.12 availability?

borjamunozf commented 9 months ago

Could be this the explanation why installing azure-cli with Python 3.12 throw this error complining about pymsalruntime?

building 'pymsalruntime.pymsalruntime' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pymsalruntime
Successfully built antlr4-python3-runtime
Failed to build pymsalruntime
ERROR: Could not build wheels for pymsalruntime, which is required to install pyproject.toml-based projects
spauka commented 9 months ago

Indeed it seems like the dependency of pymsalruntime means that azure-cli is not compatible with Python 3.12. The dependancy chain seems to be: pymsalruntime<0.14,>=0.13.2 (from msal[broker]==1.24.0b2->azure-cli-core==2.53.1->azure-cli->az-cli)

The pymsalruntime library seems to use the deprecated and removed functions PyUnicode_AsUnicode and PyUnicode_FromUnicode, which means that the library cannot be built on the latest python.

mikeeq commented 9 months ago

Fedora 39 was just released, and it's containing Python 3.12. It is a huge blocker now for all Fedora users, as it's not possible to run Azure CLI "as is" without any additional hassle (running it in a container or with older version of python).

❯ az
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/__main__.py", line 39, in <module>
    az_cli = get_default_cli()
             ^^^^^^^^^^^^^^^^^
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/__init__.py", line 917, in get_default_cli
    from azure.cli.core.azlogging import AzCliLogging
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/azlogging.py", line 30, in <module>
    from azure.cli.core.commands.events import EVENT_INVOKER_PRE_CMD_TBL_TRUNCATE
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 25, in <module>
    from azure.cli.core.extension import get_extension
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/extension/__init__.py", line 11, in <module>
    from distutils.sysconfig import get_python_lib  # pylint: disable=deprecated-module
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils'
❯ uname -a
Linux 6.5.10-300.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Nov  2 20:01:06 UTC 2023 x86_64 GNU/Linux
❯ python --version
Python 3.12.0

Links:

SumedhSingh12 commented 9 months ago

Do we have an ETA for this? This is blocking our customers who are using python 3.12 from using our azure cli extension.

jiasli commented 9 months ago

The pymsalruntime library seems to use the deprecated and removed functions PyUnicode_AsUnicode and PyUnicode_FromUnicode, which means that the library cannot be built on the latest python.

I have created https://github.com/AzureAD/microsoft-authentication-library-for-python/issues/624 to track this issue.

jiasli commented 9 months ago

Do we have an ETA for this? This is blocking our customers who are using python 3.12 from using our azure cli extension.

While we will plan this feature as soon as possible, we can't provide an ETA at the moment.

Azure CLI is a product, not a library, so only MSI, DEB, RPM packages and docker images released by Microsoft are officially supported: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli.

Also, in the source code, there is no declaration that Python 3.12 is supported:

https://github.com/Azure/azure-cli/blob/f74a18bc269b41b1b9d2712113ac54cba16b7e02/src/azure-cli/setup.py#L39-L50

We don't guarantee Azure CLI works on all Python versions, such as the latest Python 3.12 or the deprecated Python 3.6. Installing Azure CLI with pip (https://github.com/Azure/azure-cli/issues/20476) is only a supplemental feature we provide for unsupported platforms, but without any warranty.

Thanks for understanding.

MrMarkPower commented 7 months ago

@jiasli its now over 2 months since this issue was first raised, are you any closer to knowing when this feature release (i.e. support for python 3.12) will be available please?

mkleinbort-ic commented 6 months ago

Any updates?

AkechiShiro commented 6 months ago

@mkleinbort-ic there is a workaround to install this dependency : https://github.com/AzureAD/microsoft-authentication-library-for-python/issues/624#issuecomment-1933241011

But I haven't tested it yet, I don't have the time to meddle with this at the moment.

DMiradakis commented 5 months ago

The suggested workaround from https://github.com/Azure/azure-cli/issues/27673#issuecomment-1942158403 above did not work for me. However, simply installing the setuptools library did the trick.

Python version: 3.12 pip install setuptools

cbassett-silverstream commented 5 months ago

The suggested workaround from #27673 (comment) above did not work for me. However, simply installing the setuptools library did the trick.

Python version: 3.12 pip install setuptools

Sadly, this didn't work for me either (I'm on Windows). Same error as before. Thanks for sharing though.

ddl-kfrench commented 5 months ago

hey -- just wanted to bump this issue again to see if there is any estimate that can be provided for when (or if) the azure CLI will be compatible with python 3.12.

PeterBosch2 commented 5 months ago

I saw that somebody states above that azure-cli does not support the installation via pip for Python 3.12. So, this message is just to spread the information, but no action is required.

A problem arose for the azure-cli feature in the DevContainer setup, as Debian Bookworm was not in the list of native installations, and a fallback to go via pip popped in. -> https://github.com/devcontainers/features/issues/899

As I understand, the problem will be fixed over there, adding bookworm to the list of native installations.

MrMarkPower commented 5 months ago

Just to provide quick update re: latest release of azure-cli i.e. version 2.58.0 (5th March 2024) This version of azure-cli still fails to install on Windows with the same pymsalruntime fatal error LNK1120 errors that were reported back in November 2023

C:\windows\system32>pip install azure-cli --upgrade Collecting azure-cli Downloading azure_cli-2.58.0-py3-none-any.whl.metadata (8.4 kB) ... Collecting pymsalruntime<0.14,>=0.13.2 (from msal[broker]==1.26.0->azure-cli-core==2.58.0->azure-cli) Using cached pymsalruntime-0.13.12.tar.gz (1.1 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done ... Building wheels for collected packages: pymsalruntime Building wheel for pymsalruntime (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for pymsalruntime (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [50 lines of output] ... "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /Tcpymsalruntime/PyMsalRuntime.c /Fobuild\temp.win-amd64-cpython-312\Release\pymsalruntime/PyMsalRuntime.obj PyMsalRuntime.c pymsalruntime/PyMsalRuntime.c(2888): warning C4013: 'PyUnicode_FromUnicode' undefined; assuming extern returning int pymsalruntime/PyMsalRuntime.c(2888): warning C4047: '=': 'PyObject *' differs in levels of indirection from 'int' pymsalruntime/PyMsalRuntime.c(3230): warning C4013: 'PyUnicode_AsUnicode' undefined; assuming extern returning int ... Creating library build\temp.win-amd64-cpython-312\Release\pymsalruntime\pymsalruntime.cp312-win_amd64.lib and object build\temp.win-amd64-cpython-312\Release\pymsalruntime\pymsalruntime.cp312-win_amd64.exp PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_AsUnicode PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_FromUnicode build\lib.win-amd64-cpython-312\pymsalruntime\pymsalruntime.cp312-win_amd64.pyd : fatal error LNK1120: 2 unresolved externals error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\link.exe' failed with exit code 1120

Attempting to perform a manual upgrade of dependency pymsalruntime... pip install pymsalruntime --upgrade and pip install pymsalruntime==0.14.0 also fails with the same error. I see the latest release (0.14.1) was yanked back in January 2024 https://pypi.org/project/pymsalruntime/0.14.1/

rakotomandimby commented 3 months ago

pip install setuptools made Azure CLI work on my workstation, though I did not depply test. I just made simple az invocation tests, such as --version and so on.

MrMarkPower commented 2 months ago

Finally found some time to return to this and have thankfully made some progress. I've installed the latest pymsalruntime which was released last month... https://pypi.org/project/pymsalruntime/0.14.2/ pip install pymsalruntime==0.14.2 and the current release of azure_cli which was release 9 days ago... https://pypi.org/project/azure-cli/ pip install azure-cli==2.61.0 and azure_cli has finally installed successfully with python 3.12 on Windows Server 2022 Thank you to all involved in sorting this out

onlined commented 2 months ago

@bebound can you also add microsoft/knack#279 to TODO list at description?

ringerc commented 2 months ago

This breaks Azure CLI on Ubuntu 24.04 LTS (Noble Numbat) when using the system Python, e.g. installing with pipx. The install succeeds, but the component fails at runtime as it fails to declare a needed dependency:

...
    from distutils.sysconfig import get_python_lib  # pylint: disable=deprecated-module
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils'

If you installed the Azure CLI with pipx (pipx install azure-cli), you can

pipx inject azure-cli setuptools

to work around this.

It looks like the package fails to declare a dependency on setuptools.

jiasli commented 1 month ago

The latest pymsalruntime now supports Python 3.12: https://pypi.org/project/pymsalruntime/0.16.2/#files

image

Goldziher commented 1 month ago

Is there already work to ensure compatibility with python 3.13?

jiasli commented 3 weeks ago

Is there already work to ensure compatibility with python 3.13?

Feel free to create a new issue.