search

pypa / setuptools

Official project repository for the Setuptools build system
https://pypi.org/project/setuptools/
MIT License
2.52k stars 1.19k forks source link

MSVC build tools not recognized #4024

Closed AmberElferink closed 1 year ago

AmberElferink commented 1 year ago

setuptools version

68.0.0

Python version

3.8.17

OS

Windows 11

Additional environment information

I installed Visual Studio 2019 and 2022. In both 2019 and 2022 I installed MSVC v140 - VS 2015 C++ build tools (v14.00) AND MSVC v142 - VS 2019 C++ x64/x86 build tools (v14.29-16.10) AND (in 2022): MSVC v143 - VS 2022 C++ x64/x86 build tools (v14.37-17.7)

Example: image

Description

Compiling a default hello world script with setuptools in python gives an error not recognizing MSVC build tools, while the build tools are installed for all visual studio versions.

Expected behavior

I expect the script to see the Build Tools I installed correctly.

How to Reproduce

# Available at setup time due to pyproject.toml
from pybind11.setup_helpers import Pybind11Extension, build_ext
from setuptools import setup

__version__ = "0.0.1"

# The main interface is through Pybind11Extension.
# * You can add cxx_std=11/14/17, and then build_ext can be removed.
# * You can set include_pybind11=false to add the include directory yourself,
#   say from a submodule.
#
# Note:
#   Sort input source files if you glob sources to ensure bit-for-bit
#   reproducible builds (https://github.com/pybind/python_example/pull/53)

ext_modules = [
    Pybind11Extension("SGcvNORM",
        ["src/main.cpp"],
        # Example: passing in the version to the compiled code
        define_macros = [('VERSION_INFO', __version__)],
        ),
]

setup(
    name="SGcvNORM",
    version=__version__,
    description="A test project using pybind11",
    long_description="",
    ext_modules=ext_modules,
    # extras_require={"test": "pytest"},
    # Currently, build_ext only provides an optional "highest supported C++
    # level" feature, but in the future it may provide more features.
    cmdclass={"build_ext": build_ext},
    zip_safe=False,
    python_requires=">=3.7",
)

Output

The error I get is:

(NovaDetection) C:\CodeRepositories\SenseGlove\Python_Implementation\CPP_libraries>pip install SGcvHD/Installation_folder
Processing c:\coderepositories\senseglove\python_implementation\cpp_libraries\sgcvhd\installation_folder
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: SGcvHD
  Building wheel for SGcvHD (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'SGcvHD' extension
      VSWHERE: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe
      FANCY PATH:
      FANCY PATH after joining  VC\Auxiliary\Build
      VSWHERE: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe
      FANCY PATH:
      FANCY PATH after joining  VC\Auxiliary\Build
      creating build\temp.win-amd64-cpython-38
      creating build\temp.win-amd64-cpython-38\Release
      creating build\temp.win-amd64-cpython-38\Release\src
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DVERSION_INFO=0.0.1 -IC:\Users\eempi\anaconda3\envs\NovaDetection\lib\site-packages\pybind11\include -IC:\Users\eempi\anaconda3\envs\NovaDetection\include -IC:\Users\eempi\anaconda3\envs\NovaDetection\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" /EHsc /Tpsrc/main.cpp /Fobuild\temp.win-amd64-cpython-38\Release\src/main.obj /std:c++latest /EHsc /bigobj
      main.cpp
      c:\users\eempi\anaconda3\envs\novadetection\lib\site-packages\pybind11\include\pybind11\detail/common.h(147): fatal error C1189: #error:  pybind11 2.10+ requires MSVC 2017 or newer
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for SGcvHD

I tried to find out where it is coming from, so put some prints in the msvc.py of the setuptools package (which are the FANCY PATH prints in the error above):

      suitable_components = (
        "Microsoft.VisualStudio.Component.VC.Tools.x86.x64",
        "Microsoft.VisualStudio.Workload.WDExpress",
    )

    for component in suitable_components:
        # Workaround for `-requiresAny` (only available on VS 2017 > 15.6)
        with contextlib.suppress(CalledProcessError, OSError, UnicodeDecodeError):
            path = (
                subprocess.check_output(
                    [
                        join(
                            root, "Microsoft Visual Studio", "Installer", "vswhere.exe"
                        ),
                        "-latest",
                        "-prerelease",
                        "-requires",
                        component,
                        "-property",
                        "installationPath",
                        "-products",
                        "*",
                    ]
                )
                .decode(encoding="mbcs", errors="strict")
                .strip()
            )
            print("VSWHERE:", join(
                            root, "Microsoft Visual Studio", "Installer", "vswhere.exe"
                        ))
            print("FANCY PATH: ", path)

            path = join(path, "VC", "Auxiliary", "Build")
            print("FANCY PATH after joining ", path)
            if isdir(path):
                return 15, path

It seems vswhere.exe is spitting out no result.

Note, there also is a Visual Studio 14.0 folder on my PC (related to Visual Studio 2015), from which is the cl.exe is apparently running right now. If that folder is deleted, it gives:

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'SGcvHD' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

Running vswhere.exe manually gives:

 C:\Program Files (x86)\Microsoft Visual Studio\Installer>vswhere -products *
 Visual Studio Locator version 3.1.4+c7a417bfb4 [query version 3.7.2174.19405]
Copyright (C) Microsoft Corporation. All rights reserved.

instanceId: 4abbfbbf
installDate: 23-6-2023 11:06:40
installationName: VisualStudio/16.11.27+33801.447
installationPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
installationVersion: 16.11.33801.447
productId: Microsoft.VisualStudio.Product.Community
productPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\devenv.exe
state: 4294967295
isComplete: 1
isLaunchable: 1
isPrerelease: 0
isRebootRequired: 0
displayName: Visual Studio Community 2019
description: Powerful IDE, free for students, open-source contributors, and individuals
channelId: VisualStudio.16.Release
channelUri: https://aka.ms/vs/16/release/channel
enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service
installedChannelId: VisualStudio.16.Release
installedChannelUri: https://aka.ms/vs/16/release/channel
releaseNotes: https://docs.microsoft.com/en-us/visualstudio/releases/2019/release-notes-v16.11#16.11.27
thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=660909
updateDate: 2023-06-23T09:06:40.5468646Z
catalog_buildBranch: d16.11
catalog_buildVersion: 16.11.33801.447
catalog_id: VisualStudio/16.11.27+33801.447
catalog_localBuild: build-lab
catalog_manifestName: VisualStudio
catalog_manifestType: installer
catalog_productDisplayVersion: 16.11.27
catalog_productLine: Dev16
catalog_productLineVersion: 2019
catalog_productMilestone: RTW
catalog_productMilestoneIsPreRelease: False
catalog_productName: Visual Studio
catalog_productPatchVersion: 27
catalog_productPreReleaseMilestoneSuffix: 1.0
catalog_productSemanticVersion: 16.11.27+33801.447
catalog_requiredEngineVersion: 2.11.72.18200
properties_campaignId:
properties_channelManifestId: VisualStudio.16.Release/16.11.27+33801.447
properties_nickname:
properties_setupEngineFilePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\setup.exe

instanceId: a8a4379a
installDate: 13-7-2023 14:26:55
installationName: VisualStudio/17.7.2+34018.315
installationPath: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools
installationVersion: 17.7.34018.315
productId: Microsoft.VisualStudio.Product.BuildTools
productPath: C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\Tools\LaunchDevCmd.bat
state: 4294967295
isComplete: 1
isLaunchable: 1
isPrerelease: 0
isRebootRequired: 0
displayName: Visual Studio Build Tools 2022
description: The Visual Studio Build Tools allows you to build native and managed MSBuild-based applications without requiring the Visual Studio IDE. There are options to install the Visual C++ compilers and libraries, MFC, ATL, and C++/CLI support.
channelId: VisualStudio.17.Release
channelUri: https://aka.ms/vs/17/release/channel
enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service
installedChannelId: VisualStudio.17.Release
installedChannelUri: https://aka.ms/vs/17/release/channel
releaseNotes: https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-notes-v17.7#17.7.2
thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=661288
updateDate: 2023-08-23T11:45:46.4227837Z
catalog_buildBranch: d17.7
catalog_buildVersion: 17.7.34018.315
catalog_id: VisualStudio/17.7.2+34018.315
catalog_localBuild: build-lab
catalog_manifestName: VisualStudio
catalog_manifestType: installer
catalog_productDisplayVersion: 17.7.2
catalog_productLine: Dev17
catalog_productLineVersion: 2022
catalog_productMilestone: RTW
catalog_productMilestoneIsPreRelease: False
catalog_productName: Visual Studio
catalog_productPatchVersion: 2
catalog_productPreReleaseMilestoneSuffix: 1.0
catalog_productSemanticVersion: 17.7.2+34018.315
catalog_requiredEngineVersion: 3.7.2175.60206
properties_campaignId:
properties_channelManifestId: VisualStudio.17.Release/17.7.2+34018.315
properties_nickname:
properties_setupEngineFilePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\setup.exe

So vswhere does show some Build Tools for 2022 installed, but apparently they are not correctly seen as output by the python script? Can this be fixed?

abravalheri commented 1 year ago

Hi @AmberElferink, what is the output of vswhere when you run it manually with the same flags used by setuptools?

(e.g.

vswhere -latest -prerelease -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath -products *
vswhere -latest -prerelease -requires Microsoft.VisualStudio.Workload.WDExpress -property installationPath -products *

)

AmberElferink commented 1 year ago

Hi @abravalheri, Thanks for picking it up so quickly. Both of those commands return empty

abravalheri commented 1 year ago

Hi @AmberElferink, I think you need to ensure your installation contains either Microsoft.VisualStudio.Component.VC.Tools.x86.x64 or Microsoft.VisualStudio.Workload.WDExpress (these seem to be the components setuptools depends to be able to compile the code).

AmberElferink commented 1 year ago

Can you tell me how I can get these? I did install the packages I mentioned at the top of the post. Is there somewhere else I should be looking for?

abravalheri commented 1 year ago

If you already tried to download https://visualstudio.microsoft.com/visual-cpp-build-tools/ (link in the error message) and it failed, I am not sure how else you can obtain it...

Maybe you need to select the option with "C++" in the installers' GUI related to Workload?

If you already are doing that and vswhere keeps returning an empty result for the commands above maybe the best would be asking the maintainers of vswhere (probably https://github.com/microsoft/vswhere?)

AmberElferink commented 1 year ago

Thanks, indeed I selected the full C++ package in Visual Studio 2019 now and then it was able to find it. A bit over the top, but if it works, it works :)