search

microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
22.86k stars 6.31k forks source link

Python 3 installation failure #33140

Closed jahmezz closed 1 year ago

jahmezz commented 1 year ago

I am attempting to install Python 3 through vcpkg.

My computer is 64-bit Windows 11. I have Visual Studio 2017.

Here is the output.

`D:\vcpkg>vcpkg install python3 --triplet x64-windows Computing installation plan... A suitable version of cmake was not found (required v3.27.1) Downloading portable cmake 3.27.1... Downloading cmake... https://github.com/Kitware/CMake/releases/download/v3.27.1/cmake-3.27.1-windows-i386.zip->D:\vcpkg\downloads\cmake-3.27.1-windows-i386.zip Downloading https://github.com/Kitware/CMake/releases/download/v3.27.1/cmake-3.27.1-windows-i386.zip Extracting cmake... The following packages will be built and installed: python3:x64-windows -> 3.11.4#1 Detecting compiler hash for triplet x64-windows... A suitable version of powershell-core was not found (required v7.2.11) Downloading portable powershell-core 7.2.11... Downloading powershell-core... https://github.com/PowerShell/PowerShell/releases/download/v7.2.11/PowerShell-7.2.11-win-x86.zip->D:\vcpkg\downloads\PowerShell-7.2.11-win-x86.zip Downloading https://github.com/PowerShell/PowerShell/releases/download/v7.2.11/PowerShell-7.2.11-win-x86.zip Extracting powershell-core... A suitable version of 7zip was not found (required v21.7.0) Downloading portable 7zip 21.7.0... Downloading 7zip... https://www.7-zip.org/a/7z2107-extra.7z->D:\vcpkg\downloads\7z2107-extra.7z Downloading https://www.7-zip.org/a/7z2107-extra.7z Extracting 7zip... Restored 0 package(s) from C:\Users\jameskahng\AppData\Local\vcpkg\archives in 317 us. Use --debug to see more details. Installing 1/1 python3:x64-windows... Building python3:x64-windows... -- Downloading https://github.com/python/cpython/archive/v3.11.4.tar.gz -> python-cpython-v3.11.4.tar.gz... -- Extracting source D:/vcpkg/downloads/python-cpython-v3.11.4.tar.gz -- Applying patch 0001-only-build-required-projects.patch -- Applying patch 0003-use-vcpkg-zlib.patch -- Applying patch 0004-devendor-external-dependencies.patch -- Applying patch 0005-dont-copy-vcruntime.patch -- Applying patch 0008-python.pc.patch -- Applying patch 0010-dont-skip-rpath.patch -- Applying patch 0012-force-disable-curses.patch -- Applying patch 0014-fix-get-python-inc-output.patch -- Applying patch 0015-dont-use-WINDOWS-def.patch -- Using source at D:/vcpkg/buildtrees/python3/src/v3.11.4-3f8cd2cee7.clean -- Downloading https://www.python.org/ftp/python/3.11.4/python-3.11.4-embed-amd64.zip -> python-3.11.4-embed-amd64.zip... -- Getting CMake variables for x64-windows -- Building PCbuild/pcbuild.proj for Release CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:112 (message): Command failed: msbuild D:/vcpkg/buildtrees/python3/x64-windows-rel/PCbuild/pcbuild.proj /p:Configuration=Release /p:IncludeExtensions=true /p:IncludeExternals=true /p:IncludeCTypes=true /p:IncludeSSL=true /p:IncludeTkinter=false /p:IncludeTests=false /p:ForceImportBeforeCppTargets=D:/vcpkg/buildtrees/python3/src/v3.11.4-3f8cd2cee7.clean/PCbuild/python_vcpkg.props /p:IncludeUwp=false /p:_VcpkgPythonLinkage=DynamicLibrary /p:ForceImportAfterCppProps=D:/vcpkg/buildtrees/python3/x64-windows/vcpkg_msbuild.props /p:ForceImportAfterCppTargets=D:/vcpkg/buildtrees/python3/x64-windows/vcpkg_msbuild.targets /t:Rebuild /p:UseMultiToolTask=true /p:MultiProcMaxCount=13 /p:EnforceProcessCountAcrossBuilds=true /m:13 -maxCpuCount:13 /p:Platform=x64 /p:PlatformTarget=x64 /p:PlatformToolset=v141 /p:WindowsTargetPlatformVersion=10.0.19041.0 /p:VcpkgApplocalDeps=false /p:VcpkgManifestInstall=false /p:VcpkgManifestEnabled=false /p:VcpkgEnabled=false /p:VcpkgTriplet=x64-windows /p:VcpkgInstalledDir=D:/vcpkg/installed Working Directory: D:/vcpkg/buildtrees/python3/x64-windows-rel Error code: 1 See logs for more information: D:\vcpkg\buildtrees\python3\build-x64-windows-rel-out.log

Call Stack (most recent call first): installed/x64-windows/share/vcpkg-msbuild/vcpkg_msbuild_install.cmake:108 (vcpkg_execute_required_process) ports/python3/portfile.cmake:157 (vcpkg_msbuild_install) scripts/ports.cmake:147 (include)

error: building python3:x64-windows failed with: BUILD_FAILED Elapsed time to handle python3:x64-windows: 1.1 min Please ensure you're using the latest port files with git pull and vcpkg update. Then check for known issues at: https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+python3 You can submit a new issue at: https://github.com/microsoft/vcpkg/issues/new?title=[python3]+Build+error&body=Copy+issue+body+from+D%3A%2Fvcpkg%2Finstalled%2Fvcpkg%2Fissue_body.md`

Log file attached. build-x64-windows-rel-out.log

Any ideas? Thank you.

JackBoosY commented 1 year ago

fatal error LNK1112: module machine type 'x86' conflicts with target machine type 'x64' _"D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild\obj\311win32_Release_freezemodule\traceback.obj"

Not sure why select the x86 configuration.

Osyotr commented 1 year ago

This part of patch should've fixed the error but it is guarded by if(VCPKG_CROSSCOMPILING): https://github.com/microsoft/vcpkg/pull/33026/files#diff-b1c9dc64ec61156b63eb032c9c780c4b2906860664cd661115fae016efc50d8aR30

My guess is that PreferredToolArchitecture is x86 on VS2019, but we pass x64 flags to it. @Neumann-A can this patch be applied unconditionally?

Neumann-A commented 1 year ago

can this patch be applied unconditional?

probably. Never tested it for another use case however.

LilyWangLL commented 1 year ago

My computer is 64-bit Windows 11.

       ClCompile:
         C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x86\CL.exe

It's strange, why is your computer 64-bit but VS chose HostX86?

You can modify the following codes in the portfile.cmake file of python3, change if(VCPKG_CROSSCOMPILING) to if(1) to apply the patch 0016-fix-win-cross.patch, and then rebuild python3.

    if(VCPKG_CROSSCOMPILING)
        list(APPEND PATCHES "0016-fix-win-cross.patch")
    endif()
jahmezz commented 1 year ago

I went to the portfile.cmake at D:\vcpkg\buildtrees\versioning_\versions\python3\531061b4485adebb80364473d227b0edf4ee8ed9 and updated the if statement to if(1), which lets patch 16 to be included.

Output: Detecting compiler hash for triplet x64-windows... The following packages will be built and installed: python3:x64-windows -> 3.11.4#1 -- D:\vcpkg\buildtrees\versioning\versions\python3\531061b4485adebb80364473d227b0edf4ee8ed9 Restored 0 package(s) from C:\Users\jameskahng\AppData\Local\vcpkg\archives in 228 us. Use --debug to see more details. Installing 1/1 python3:x64-windows... Building python3:x64-windows... -- Installing port from location: D:\vcpkg\buildtrees\versioning\versions\python3\531061b4485adebb80364473d227b0edf4ee8ed9 -- Using cached python-cpython-v3.11.4.tar.gz. -- Cleaning sources at D:/vcpkg/buildtrees/python3/src/v3.11.4-864ed8c023.clean. Use --editable to skip cleaning for the packages you specify. -- Extracting source D:/vcpkg/downloads/python-cpython-v3.11.4.tar.gz -- Applying patch 0001-only-build-required-projects.patch -- Applying patch 0003-use-vcpkg-zlib.patch -- Applying patch 0004-devendor-external-dependencies.patch -- Applying patch 0005-dont-copy-vcruntime.patch -- Applying patch 0008-python.pc.patch -- Applying patch 0010-dont-skip-rpath.patch -- Applying patch 0012-force-disable-curses.patch -- Applying patch 0014-fix-get-python-inc-output.patch -- Applying patch 0015-dont-use-WINDOWS-def.patch -- Applying patch 0016-fix-win-cross.patch -- Using source at D:/vcpkg/buildtrees/python3/src/v3.11.4-864ed8c023.clean -- Getting CMake variables for x64-windows -- Building PCbuild/pcbuild.proj for Release CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:112 (message): Command failed: msbuild D:/vcpkg/buildtrees/python3/x64-windows-rel/PCbuild/pcbuild.proj /p:Configuration=Release /p:IncludeExtensions=true /p:IncludeExternals=true /p:IncludeCTypes=true /p:IncludeSSL=true /p:IncludeTkinter=false /p:IncludeTests=false /p:ForceImportBeforeCppTargets=D:/vcpkg/buildtrees/python3/src/v3.11.4-864ed8c023.clean/PCbuild/python_vcpkg.props /p:IncludeUwp=false /p:_VcpkgPythonLinkage=DynamicLibrary /p:ForceImportAfterCppProps=D:/vcpkg/buildtrees/python3/x64-windows/vcpkg_msbuild.props /p:ForceImportAfterCppTargets=D:/vcpkg/buildtrees/python3/x64-windows/vcpkg_msbuild.targets /t:Rebuild /p:UseMultiToolTask=true /p:MultiProcMaxCount=13 /p:EnforceProcessCountAcrossBuilds=true /m:13 -maxCpuCount:13 /p:Platform=x64 /p:PlatformTarget=x64 /p:PlatformToolset=v141 /p:WindowsTargetPlatformVersion=10.0.19041.0 /p:VcpkgApplocalDeps=false /p:VcpkgManifestInstall=false /p:VcpkgManifestEnabled=false /p:VcpkgEnabled=false /p:VcpkgTriplet=x64-windows /p:VcpkgInstalledDir=D:/vcpkg/vcpkg_installed Working Directory: D:/vcpkg/buildtrees/python3/x64-windows-rel Error code: 1 See logs for more information: D:\vcpkg\buildtrees\python3\build-x64-windows-rel-out.log

Call Stack (most recent call first): vcpkg_installed/x64-windows/share/vcpkg-msbuild/vcpkg_msbuild_install.cmake:108 (vcpkg_execute_requiredprocess) buildtrees/versioning/versions/python3/531061b4485adebb80364473d227b0edf4ee8ed9/portfile.cmake:157 (vcpkg_msbuild_install) scripts/ports.cmake:147 (include)

error: building python3:x64-windows failed with: BUILD_FAILED Elapsed time to handle python3:x64-windows: 1.2 min Please ensure you're using the latest port files with git pull and vcpkg update. Then check for known issues at: https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+python3 You can submit a new issue at: https://github.com/microsoft/vcpkg/issues/new?title=[python3]+Build+error&body=Copy+issue+body+from+D%3A%2Fvcpkg%2Fvcpkg_installed%2Fvcpkg%2Fissue_body.md

There is now a different error message in the logs. "D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild\pcbuild.proj" (Rebuild target) (1) -> "D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild_freeze_module.vcxproj" (Build target) (28:2) -> cl : Command line warning D9025: overriding '/Od' with '/O2' [D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild_freeze_module.vcxproj] cl : Command line warning D9025: overriding '/Zi' with '/Z7' [D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild_freeze_module.vcxproj]

   "D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild\pcbuild.proj" (Rebuild target) (1) ->
   "D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild\_freeze_module.vcxproj" (Build target) (28:2) ->
   (_RebuildFrozen target) -> 
     D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild\_freeze_module.vcxproj(381,5): error MSB3073: The command ""_freeze_module.exe" "importlib._bootstrap" "D:\vcpkg\buildtrees\python3\x64-windows-rel\Lib\importlib\_bootstrap.py" "D:\vcpkg\buildtrees\python3\x64-windows-rel\PCbuild\obj\311amd64_Release\_freeze_module\importlib._bootstrap.g.h"" exited with code 9009.

28 Warning(s)
1 Error(s)

Full message log attached. build-x64-windows-rel-out.log

jalegido commented 1 year ago

jahmezz, could you test this patch in your machine? I had same problem, and this patch has solved it.

jahmezz commented 1 year ago

Hello,

I am unclear about exactly how to test the patch on my machine, but what I did was edit that cmake file again (in the versioning folder) to load the patch, copied the patch file into the folder, and attempted install again.

It did not work.

I am attaching D:\vcpkg\buildtrees\python3\src\v3.11.4-3f8cd2cee7.clean\PCbuild_freeze_module.vcxproj to perhaps highlight any differences between our copies. (At bottom of this post.)

Output Detecting compiler hash for triplet x64-windows... The following packages will be built and installed: python3:x64-windows -> 3.11.4#1 -- D:\vcpkg\buildtrees\versioning\versions\python3\531061b4485adebb80364473d227b0edf4ee8ed9 Restored 0 package(s) from C:\Users\jameskahng\AppData\Local\vcpkg\archives in 116 us. Use --debug to see more details. Installing 1/1 python3:x64-windows... Building python3:x64-windows... -- Installing port from location: D:\vcpkg\buildtrees\versioning\versions\python3\531061b4485adebb80364473d227b0edf4ee8ed9 -- Using cached python-cpython-v3.11.4.tar.gz. -- Extracting source D:/vcpkg/downloads/python-cpython-v3.11.4.tar.gz -- Applying patch 0001-only-build-required-projects.patch -- Applying patch 0003-use-vcpkg-zlib.patch -- Applying patch 0004-devendor-external-dependencies.patch -- Applying patch 0005-dont-copy-vcruntime.patch -- Applying patch 0008-python.pc.patch -- Applying patch 0010-dont-skip-rpath.patch -- Applying patch 0012-force-disable-curses.patch -- Applying patch 0014-fix-get-python-inc-output.patch -- Applying patch 0015-dont-use-WINDOWS-def.patch -- Applying patch 0016-fix-win-cross.patch -- Applying patch 0017-fix-win.patch CMake Error at scripts/cmake/z_vcpkg_apply_patches.cmake:34 (message): Applying patch failed: Checking patch PCbuild/_freeze_module.vcxproj...

error: while searching for:

<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<Target Name="_RebuildGetPath" AfterTargets="_RebuildFrozen" Condition="$(Configuration) != 'PGUpdate'">
  <Exec Command='"$(TargetPath)" "%(GetPath.ModName)" "%(GetPath.FullPath)" "%(GetPath.IntFile)"' />

  <Copy SourceFiles="%(GetPath.IntFile)"
        DestinationFiles="%(GetPath.OutFile)"

error: patch failed: PCbuild/_freeze_module.vcxproj:366

error: PCbuild/_freeze_module.vcxproj: patch does not apply

Checking patch PCbuild/pcbuild.proj...

error: while searching for:

<ItemDefinitionGroup>?
  <FreezeProjects>?
    <Platform>$(PreferredToolArchitecture)</Platform>?
    <Configuration>$(Configuration)</Configuration>?
    <Configuration Condition="$(Configuration) == 'PGInstrument'">Release</Configuration>?
    <Properties></Properties>?

error: patch failed: PCbuild/pcbuild.proj:14

error: PCbuild/pcbuild.proj: patch does not apply

Call Stack (most recent call first): scripts/cmake/vcpkg_extract_source_archive.cmake:153 (z_vcpkg_apply_patches) scripts/cmake/vcpkg_extract_source_archive_ex.cmake:8 (vcpkg_extract_source_archive) scripts/cmake/vcpkg_from_github.cmake:113 (vcpkg_extract_source_archiveex) buildtrees/versioning/versions/python3/531061b4485adebb80364473d227b0edf4ee8ed9/portfile.cmake:54 (vcpkg_from_github) scripts/ports.cmake:147 (include)

error: building python3:x64-windows failed with: BUILD_FAILED Elapsed time to handle python3:x64-windows: 4.8 s Please ensure you're using the latest port files with git pull and vcpkg update. Then check for known issues at: https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+python3 You can submit a new issue at: https://github.com/microsoft/vcpkg/issues/new?title=[python3]+Build+error&body=Copy+issue+body+from+D%3A%2Fvcpkg%2Fvcpkg_installed%2Fvcpkg%2Fissue_body.md

_freeze_module.vcxproj

<?xml version="1.0" encoding="utf-8"?>

Debug ARM Debug ARM64 Debug Win32 Debug x64 PGInstrument ARM PGInstrument ARM64 PGInstrument Win32 PGInstrument x64 PGUpdate ARM PGUpdate ARM64 PGUpdate Win32 PGUpdate x64 Release ARM Release ARM64 Release Win32 Release x64 {19C0C13F-47CA-4432-AFF3-799A296A4DDC} Win32Proj _freeze_module false Application Unicode Py_NO_ENABLE_SHARED;Py_BUILD_CORE;_CONSOLE;%(PreprocessorDefinitions) Disabled false Console version.lib;ws2_32.lib;pathcch.lib;bcrypt.lib;%(AdditionalDependencies) Default VPATH="$(PyVPath)";%(PreprocessorDefinitions) importlib._bootstrap $(IntDir)importlib._bootstrap.g.h $(PySourcePath)Python\frozen_modules\importlib._bootstrap.h importlib._bootstrap_external $(IntDir)importlib._bootstrap_external.g.h $(PySourcePath)Python\frozen_modules\importlib._bootstrap_external.h zipimport $(IntDir)zipimport.g.h $(PySourcePath)Python\frozen_modules\zipimport.h abc $(IntDir)abc.g.h $(PySourcePath)Python\frozen_modules\abc.h codecs $(IntDir)codecs.g.h $(PySourcePath)Python\frozen_modules\codecs.h io $(IntDir)io.g.h $(PySourcePath)Python\frozen_modules\io.h _collections_abc $(IntDir)_collections_abc.g.h $(PySourcePath)Python\frozen_modules\_collections_abc.h _sitebuiltins $(IntDir)_sitebuiltins.g.h $(PySourcePath)Python\frozen_modules\_sitebuiltins.h genericpath $(IntDir)genericpath.g.h $(PySourcePath)Python\frozen_modules\genericpath.h ntpath $(IntDir)ntpath.g.h $(PySourcePath)Python\frozen_modules\ntpath.h posixpath $(IntDir)posixpath.g.h $(PySourcePath)Python\frozen_modules\posixpath.h os $(IntDir)os.g.h $(PySourcePath)Python\frozen_modules\os.h site $(IntDir)site.g.h $(PySourcePath)Python\frozen_modules\site.h stat $(IntDir)stat.g.h $(PySourcePath)Python\frozen_modules\stat.h importlib.util $(IntDir)importlib.util.g.h $(PySourcePath)Python\frozen_modules\importlib.util.h importlib.machinery $(IntDir)importlib.machinery.g.h $(PySourcePath)Python\frozen_modules\importlib.machinery.h runpy $(IntDir)runpy.g.h $(PySourcePath)Python\frozen_modules\runpy.h __hello__ $(IntDir)__hello__.g.h $(PySourcePath)Python\frozen_modules\__hello__.h __phello__ $(IntDir)__phello__.g.h $(PySourcePath)Python\frozen_modules\__phello__.h __phello__.ham $(IntDir)__phello__.ham.g.h $(PySourcePath)Python\frozen_modules\__phello__.ham.h __phello__.ham.eggs $(IntDir)__phello__.ham.eggs.g.h $(PySourcePath)Python\frozen_modules\__phello__.ham.eggs.h __phello__.spam $(IntDir)__phello__.spam.g.h $(PySourcePath)Python\frozen_modules\__phello__.spam.h frozen_only $(IntDir)frozen_only.g.h $(PySourcePath)Python\frozen_modules\frozen_only.h getpath $(IntDir)getpath.g.h $(PySourcePath)Python\frozen_modules\getpath.h
jahmezz commented 1 year ago

@jalegido Found the issue after some tinkering.

Patch 17 was written based on code pre-patch 16.

Patch 16 causes the proposed patch 17 to become invalid.

I updated the cmake file to skip patch 16 and the proposed patch 17 allowed my build to complete!

Required adjustment Write patch 17 to logically proceed from the codebase after application of Patch 16.

jalegido commented 1 year ago

Hello, you can see that patch 17 is in an "else" statement. So you only apply patch 17 if "patch 16" is not applyed. This is the reason both patch are independent.

jahmezz commented 1 year ago

@jalegido Ah! My mistake. Applied and verified to be completing properly.

Patch 15 went to Patch 17. Build completed.