search

AzureAD / microsoft-authentication-library-for-python

Microsoft Authentication Library (MSAL) for Python makes it easy to authenticate to Microsoft Entra ID. General docs are available here https://learn.microsoft.com/entra/msal/python/ Stable APIs are documented here https://msal-python.readthedocs.io. Questions can be asked on www.stackoverflow.com with tag "msal" + "python".
https://stackoverflow.com/questions/tagged/azure-ad-msal+python
Other
816 stars 203 forks source link

Installing `msal[broker]` fails on Python 3.12: unresolved external symbol PyUnicode_AsUnicode, PyUnicode_FromUnicode #624

Closed jiasli closed 7 months ago

jiasli commented 12 months ago

Describe the bug Even though #604 declared the support for Python 3.12, installing msal[broker] fails on Python 3.12.

As pymsalruntime doesn't have a GitHub repo, the MSAL repo is the closest to pymsalruntime.

To Reproduce

C:\Users\xxx\AppData\Local\Programs\Python\Python312\python.exe -m venv py312
.\py312\Scripts\Activate.ps1
python.exe -m pip install --upgrade pip
pip install msal[broker]

Error:

Building wheels for collected packages: pymsalruntime
  Building wheel for pymsalruntime (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pymsalruntime (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [50 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-312
      creating build\lib.win-amd64-cpython-312\pymsalruntime
      copying pymsalruntime\__init__.py -> build\lib.win-amd64-cpython-312\pymsalruntime
      copying pymsalruntime\msalruntime.dll -> build\lib.win-amd64-cpython-312\pymsalruntime
      copying pymsalruntime\PyMsalRuntime.pyi -> build\lib.win-amd64-cpython-312\pymsalruntime
      running build_ext
      building 'pymsalruntime.pymsalruntime' extension
      creating build\temp.win-amd64-cpython-312
      creating build\temp.win-amd64-cpython-312\Release
      creating build\temp.win-amd64-cpython-312\Release\pymsalruntime
      "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Ibuild_resources -ID:\cli\py312\include -IC:\Users\xxx\AppData\Local\Programs\Python\Python312\include -IC:\Users\xxx\AppData\Local\Programs\Python\Python312\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tcpymsalruntime/PyMsalRuntime.c /Fobuild\temp.win-amd64-cpython-312\Release\pymsalruntime/PyMsalRuntime.obj
      PyMsalRuntime.c
      pymsalruntime/PyMsalRuntime.c(2888): warning C4013: 'PyUnicode_FromUnicode' undefined; assuming extern returning int
      pymsalruntime/PyMsalRuntime.c(2888): warning C4047: '=': 'PyObject *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(3230): warning C4013: 'PyUnicode_AsUnicode' undefined; assuming extern returning int
      pymsalruntime/PyMsalRuntime.c(3230): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(3267): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(3616): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(3662): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(3976): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(4325): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(4371): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(4689): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(4726): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(5040): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(5077): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(5402): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(5759): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(8287): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(8324): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(8673): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(8844): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9014): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9184): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9435): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9472): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9518): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9564): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9789): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      pymsalruntime/PyMsalRuntime.c(9826): warning C4047: '=': 'os_char *' differs in levels of indirection from 'int'
      "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:build_resources /LIBPATH:D:\cli\py312\libs /LIBPATH:C:\Users\xxx\AppData\Local\Programs\Python\Python312\libs /LIBPATH:C:\Users\xxx\AppData\Local\Programs\Python\Python312 /LIBPATH:D:\cli\py312\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64" msalruntime.lib user32.lib /EXPORT:PyInit_pymsalruntime build\temp.win-amd64-cpython-312\Release\pymsalruntime/PyMsalRuntime.obj /OUT:build\lib.win-amd64-cpython-312\pymsalruntime\pymsalruntime.cp312-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-312\Release\pymsalruntime\pymsalruntime.cp312-win_amd64.lib
         Creating library build\temp.win-amd64-cpython-312\Release\pymsalruntime\pymsalruntime.cp312-win_amd64.lib and object build\temp.win-amd64-cpython-312\Release\pymsalruntime\pymsalruntime.cp312-win_amd64.exp
      PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_AsUnicode
      PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_FromUnicode
      build\lib.win-amd64-cpython-312\pymsalruntime\pymsalruntime.cp312-win_amd64.pyd : fatal error LNK1120: 2 unresolved externals
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pymsalruntime
Failed to build pymsalruntime
ERROR: Could not build wheels for pymsalruntime, which is required to install pyproject.toml-based projects
rayluo commented 12 months ago

As pymsalruntime doesn't have a GitHub repo, the MSAL repo is the closest to pymsalruntime.

Fair enough. I'll assign this issue to @ashok672 , though. :-)

Describe the bug Even though #604 declared the support for Python 3.12, installing msal[broker] fails on Python 3.12.

604 was about MSAL Python itself being cross-Python-versions. But the msal[broker] package's dependency pymsalruntime does not currently have a build artifact for Python 3.12.

@iulico-1 , @ashok672 and I happened to mention about this known issue earlier today.

jiasli commented 12 months ago

604 was about MSAL Python itself being cross-Python-versions. But the msal[broker] package's dependency pymsalruntime does not currently have a build artifact for Python 3.12.

Well, this is only part of the reason. Even though pymsalruntime doesn't have a built artifact for Python 3.12, it should be able to compile on Python 3.12 with a C compiler (VC 2019 on my machine). The fact is it does not because of LNK2001:

      PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_AsUnicode
      PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_FromUnicode
rayluo commented 12 months ago

604 was about MSAL Python itself being cross-Python-versions. But the msal[broker] package's dependency pymsalruntime does not currently have a build artifact for Python 3.12.

Well, this is only part of the reason. Even though pymsalruntime doesn't have a built artifact for Python 3.12, it should be able to compile on Python 3.12 with a C compiler (VC 2019 on my machine). The fact is it does not because of LNK2001:

      PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_AsUnicode
      PyMsalRuntime.obj : error LNK2001: unresolved external symbol PyUnicode_FromUnicode

I heard that we ran into similar issues while attempting to build it with Python 3.12. We will work on it. By the time that we eventually solve this, we shall be able to ship an already-built whl file to PyPI, so that customers do not have to deal with all these.

AkechiShiro commented 10 months ago

Any news on this downstream issue since November 2023, just following up on this comment here : https://github.com/Azure/azure-cli/issues/27673#issuecomment-1877048735

vairakkumaar-svs-hf commented 9 months ago

It does not seems like this was uploaded to pip or the package itself fixed. Its failing in Visual Studio 2022 PY3.12 windows server 2022 x64 What is the working version combination?

rayluo commented 9 months ago

@AkechiShiro , @vairakkumaar-svs-hf ,

For now, instead of pip install msal[broker], try this as a workaround.

pip install msal
pip install pymsalruntime
rayluo commented 8 months ago

UPDATE:

jiasli commented 8 months ago

I would suggest not to close this issue until newer patch of pymsalruntime becomes available, as customers are still reporting issues in Azure CLI repo: https://github.com/Azure/azure-cli/pull/28483#issuecomment-1971723754

rayluo commented 8 months ago

UPDATE:

  • This has technically been fixed in MSAL Python 1.27

  • However, the latest PyMsalRuntime 0.14.1 happens to be not quite ready. Downstream apps would need to manually do pip install pymsalruntime==0.14.0 as a short term workaround, until newer patch of pymsalruntime becomes available.

I would suggest not to close this issue until newer patch of pymsalruntime becomes available, as customers are still reporting issues in Azure CLI repo: Azure/azure-cli#28483 (comment)

Since the problematic pymsalruntime 0.14.1 has been yanked, the existing latest MSAL Python 1.27.0 can pull the pymsalruntime 0.14.0 which works with Python 3.12. @jiasli , I would suggest:

  1. We can now close this issue (regardless of how a future pymsalruntime 0.14.x release would behave - a code base really should not keep an issue open while the actionable item is on its upstream dependency)
  2. Azure CLI can and shall start testing with PyMsalRuntime 0.14.0 now