search

microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
23.09k stars 6.37k forks source link

[python3] Support building of C extension packages with interpreter #21335

Open mtnpke opened 2 years ago

mtnpke commented 2 years ago

When installing the Python3 interpreter with vcpkg, it cannot be used to compile C extension packages. It does not find the python.h include files and (on Windows/MSVC) the python39.lib file. Error message: "Cannot open include file: 'Python.h': No such file or directory".

Is your feature request related to a problem? Please describe. Trying to install C extension packages fails, but in my opinion it should work in a "fully-fledged" Python installation.

To reproduce:

  1. Run vcpkg install python3
  2. Run <vcpkg install path>/tools/python3/python -m pip install typed-ast --no-binary ":all:"

Result:

Collecting typed-ast
  Using cached typed_ast-1.4.3.tar.gz (210 kB)
  Preparing metadata (setup.py) ... done
Skipping wheel build for typed-ast, due to binaries being disabled for it.
Installing collected packages: typed-ast
    Running setup.py install for typed-ast ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\_GIT\vcpkg\installed\x86-windows\tools\python3\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\kerling\\AppData\\Local\\Temp\\pip-install-vxynyg7q\\typed-ast_64847131d6f2482287276a3709d12278\\setup.py'"'"'; __file__='"'"'C:\\Users\\kerling\\AppData\\Local\\Temp\\pip-install-vxynyg7q\\typed-ast_64847131d6f2482287276a3709d12278\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\kerling\AppData\Local\Temp\pip-record-kmk18qw2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\_GIT\vcpkg\installed\x86-windows\tools\python3\Include\typed-ast'
         cwd: C:\Users\kerling\AppData\Local\Temp\pip-install-vxynyg7q\typed-ast_64847131d6f2482287276a3709d12278\
    Complete output (25 lines):
    running install
    running build
    running build_py
    creating build
    creating build\lib.win32-3.10
    creating build\lib.win32-3.10\typed_ast
    copying typed_ast\ast27.py -> build\lib.win32-3.10\typed_ast
    copying typed_ast\ast3.py -> build\lib.win32-3.10\typed_ast
    copying typed_ast\conversions.py -> build\lib.win32-3.10\typed_ast
    copying typed_ast\__init__.py -> build\lib.win32-3.10\typed_ast
    package init file 'ast3\tests\__init__.py' not found (or not a regular file)
    creating build\lib.win32-3.10\typed_ast\tests
    copying ast3\tests\test_basics.py -> build\lib.win32-3.10\typed_ast\tests
    running build_ext
    building '_ast27' extension
    creating build\temp.win32-3.10
    creating build\temp.win32-3.10\Release
    creating build\temp.win32-3.10\Release\ast27
    creating build\temp.win32-3.10\Release\ast27\Custom
    creating build\temp.win32-3.10\Release\ast27\Parser
    creating build\temp.win32-3.10\Release\ast27\Python
    C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\HostX86\x86\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Iast27/Include -IC:\_GIT\vcpkg\installed\x86-windows\tools\python3\include -IC:\_GIT\vcpkg\installed\x86-windows\tools\python3\Include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt /Tcast27/Custom/typed_ast.c /Fobuild\temp.win32-3.10\Release\ast27/Custom/typed_ast.obj
    typed_ast.c
    ast27/Custom/typed_ast.c(1): fatal error C1083: Cannot open include file: 'Python.h': No such file or directory
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Professional\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x86\\cl.exe' failed with exit code 2
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'C:\_GIT\vcpkg\installed\x86-windows\tools\python3\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\kerling\\AppData\\Local\\Temp\\pip-install-vxynyg7q\\typed-ast_64847131d6f2482287276a3709d12278\\setup.py'"'"'; __file__='"'"'C:\\Users\\kerling\\AppData\\Local\\Temp\\pip-install-vxynyg7q\\typed-ast_64847131d6f2482287276a3709d12278\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\kerling\AppData\Local\Temp\pip-record-kmk18qw2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\_GIT\vcpkg\installed\x86-windows\tools\python3\Include\typed-ast' Check the logs for full command output.

Proposed solution I'm not exactly sure. Either install the include and libs files also to the tools location or make sure that python correctly picks up the locations where they are stored. Maybe I'm just missing a config option?

Describe alternatives you've considered No compilation is needed when using precompiled wheels, but that is not possible in every case. I'm currently copying the headers and lib files in my CMakeLists.txt as workaround.

Additional context I'm using Windows 10 20H2. My machine is using the en-US locale. Tested with vcpkg 2021-11-02-af04ebf6274fd6f7a941bff4662b3955c64f6f42.

Tremoneck commented 2 years ago

This is a regression, with a66098ad1a9ec72de9fd4d63ecce8bd23d6b6d74 it was working correctly. After resetting the python version to 3.9.6 it works again, while leaving all other commits which have landed in the mean time alone.

connorjak commented 1 year ago

I've reproduced this on Windows 22H2 (19045.2486) with vcpkg ec7216c8e as a submodule, using this command line:

vcpkg\packages\python3_x64-windows\tools\python3\python.exe -m pip install imgui[pyglet]

It appears this is a problem with where the library compile include directories are targeted (or conversely where the headers and libs are copied).

I implemented the workaround by:

qgbcs commented 9 months ago

find_package(Python COMPONENTS Interpreter Development) error

github-actions[bot] commented 2 months ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.