pypa / setuptools

Official project repository for the Setuptools build system
https://pypi.org/project/setuptools/
MIT License
2.53k stars 1.19k forks source link

msvc.py fails with VS 2019 Build Tools 14.24 #2028

Open SofianJabal opened 4 years ago

SofianJabal commented 4 years ago

Hi,

I installed VS 2019 with Build Tools 14.24 but my "setup.py build" still fails with the below error in msvc.py file: error: Microsoft Visual C++ 14.1 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/

And AFAIK, the msvc.py should support Build Tools 14.24 What I'm missing? anyone could help?

mattip commented 4 years ago

Just hit this as well. I think it has to do with the numbering scheme for visual, which now has hundreds and not just tens, where the format string for vc_base uses %01f.

And in more detail: there is a numbering scheme to get from a msvc compiler version to a visual studio version number. Versions up to MSVC++ 14.1 _MSC_VER == 1910 used three integers, in this case 19 and 1. If you compile python yourself, the _MSC_VER string appears as part of sys.version and is parsed out to determine which version of Visual to search for. But now, they use things like MSVC++ 14.24 _MSC_VER == 1924, so the format string needs to change.

So if you compile python yourself, you will no longer be able to build c-extension modules.

SofianJabal commented 4 years ago

Thanks for the reply,

Actually I'm trying to compile pywin32 module and using python 3.8 64bit, do I should use cpython or what?

mattip commented 4 years ago

You could run the build from within the Visual Studio Command Prompt, which will set things up so python does not have to discover which compiler to use.

SofianJabal commented 4 years ago

You mean I can compile and build a python package using Visual Studio same as running "setup.py build"?

If yes، do you know where I can find instructions?

mattip commented 4 years ago

Start -> Visual Studio -> Visual Studio Command Prompt will open a DOS command window with PATHs already set up for compilation. Python should respect that and just compile your code.

SofianJabal commented 4 years ago

Thanks for the help man, I'll check and update

jaraco commented 4 years ago

Is the fix for this issue simply to replace %0.1f with %0.2f?

jaraco commented 4 years ago

Looking at the codebase, the presence of %0.1f is only in a function referencing msvc9. It looks to me like MSVC14 has its own discovery mechanism that doesn't reference that format string, so I'm not confident that code path is relevant.

jaraco commented 4 years ago

I've installed the latest build tools for VC 2017 and 2019:

image

And I believe I was able to replicate the failure:

>>> from setuptools import msvc
>>> msvc.msvc9_query_vcvarsall(14.0, 'x64')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\jaraco\AppData\Local\Temp\pip-run-l5620zvk\setuptools\msvc.py", line 140, in msvc9_query_vcvarsall
    return EnvironmentInfo(arch, ver).return_env()
  File "C:\Users\jaraco\AppData\Local\Temp\pip-run-l5620zvk\setuptools\msvc.py", line 1736, in return_env
    [self.VCIncludes,
  File "C:\Users\jaraco\AppData\Local\Temp\pip-run-l5620zvk\setuptools\msvc.py", line 1278, in VCIncludes
    return [join(self.si.VCInstallDir, 'Include'),
  File "C:\Users\jaraco\AppData\Local\Temp\pip-run-l5620zvk\setuptools\msvc.py", line 836, in VCInstallDir
    raise distutils.errors.DistutilsPlatformError(msg)
distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
jaraco commented 4 years ago

It does look like some discovery is working:

>>> msvc._msvc14_find_vc2017()
(15, 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build')
>>> msvc._msvc14_find_vcvarsall('amd64')
('C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build\\vcvarsall.bat', 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build\\..\\..\\redist\\MSVC\\14.25.28508\\onecore\\x64\\Microsoft.VC142.CRT\\vcruntime140.dll')
jaraco commented 4 years ago

The patched msvc14 function works as intended:

>>> msvc.msvc14_get_vc_env('amd64')
{'bash_completions': 'C:\\Program Files\\Git\\usr\\share\\bash-completion\\bash_completion;C:\\Program Files\\Git\\mingw64\\share\\git\\completion\\git-completion.bash', 'commandprompttype': 'Native', 'devenvdir': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\IDE\\', 'extensionsdkdir': 'C:\\Program Files (x86)\\Microsoft SDKs\\Windows Kits\\10\\ExtensionSDKs', 'framework40version': 'v4.0', 'frameworkdir': 'C:\\Windows\\Microsoft.NET\\Framework64\\', 'frameworkdir64': 'C:\\Windows\\Microsoft.NET\\Framework64\\', 'frameworkversion': 'v4.0.30319', 'frameworkversion64': 'v4.0.30319', 'include': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.25.28610\\include;C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\ucrt;C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\shared;C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\um;C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\winrt;C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\cppwinrt', 'lib': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.25.28610\\lib\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.18362.0\\ucrt\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.18362.0\\um\\x64;', 'libpath': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.25.28610\\lib\\x64;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.25.28610\\lib\\x86\\store\\references;C:\\Program Files (x86)\\Windows Kits\\10\\UnionMetadata\\10.0.18362.0;C:\\Program Files (x86)\\Windows Kits\\10\\References\\10.0.18362.0;C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319;', 'platform': 'x64', 'ucrtversion': '10.0.18362.0', 'universalcrtsdkdir': 'C:\\Program Files (x86)\\Windows Kits\\10\\', 'vcideinstalldir': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\IDE\\VC\\', 'vcinstalldir': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\', 'vctoolsinstalldir': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.25.28610\\', 'vctoolsredistdir': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Redist\\MSVC\\14.25.28508\\', 'vctoolsversion': '14.25.28610', 'visualstudioversion': '16.0', 'vs160comntools': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\Tools\\', 'vscmd_arg_app_plat': 'Desktop', 'vscmd_arg_host_arch': 'x64', 'vscmd_arg_tgt_arch': 'x64', 'vscmd_ver': '16.5.4', 'vsinstalldir': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\', 'windowslibpath': 'C:\\Program Files (x86)\\Windows Kits\\10\\UnionMetadata\\10.0.18362.0;C:\\Program Files (x86)\\Windows Kits\\10\\References\\10.0.18362.0', 'windowssdkbinpath': 'C:\\Program Files (x86)\\Windows Kits\\10\\bin\\', 'windowssdkdir': 'C:\\Program Files (x86)\\Windows Kits\\10\\', 'windowssdklibversion': '10.0.18362.0\\', 'windowssdkverbinpath': 'C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.18362.0\\', 'windowssdkversion': '10.0.18362.0\\', 'xonsh_version': '0.9.18', 'allusersprofile': 'C:\\ProgramData', 'appdata': 'C:\\Users\\jaraco\\AppData\\Roaming', 'chocolateyinstall': 'C:\\ProgramData\\chocolatey', 'chocolateylastpathupdate': 'Sat Nov  3 09:18:24 2018', 'commonprogramfiles': 'C:\\Program Files\\Common Files', 'commonprogramfiles(x86)': 'C:\\Program Files (x86)\\Common Files', 'commonprogramw6432': 'C:\\Program Files\\Common Files', 'computername': 'AKUMA', 'comspec': 'C:\\WINDOWS\\system32\\cmd.exe', 'driverdata': 'C:\\Windows\\System32\\Drivers\\DriverData', 'homedrive': 'C:', 'homepath': '\\Users\\jaraco', 'localappdata': 'C:\\Users\\jaraco\\AppData\\Local', 'logonserver': '\\\\AKUMA', 'number_of_processors': '1', 'onedrive': 'C:\\Users\\jaraco\\OneDrive', 'onedriveconsumer': 'C:\\Users\\jaraco\\OneDrive', 'os': 'Windows_NT', 'pathext': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PYW;.PY', 'processor_architecture': 'AMD64', 'processor_identifier': 'Intel64 Family 6 Model 142 Stepping 10, GenuineIntel', 'processor_level': '6', 'processor_revision': '8e0a', 'programdata': 'C:\\ProgramData', 'programfiles': 'C:\\Program Files', 'programfiles(x86)': 'C:\\Program Files (x86)', 'programw6432': 'C:\\Program Files', 'psmodulepath': 'C:\\Program Files\\WindowsPowerShell\\Modules;C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\Modules', 'public': 'C:\\Users\\Public', 'systemdrive': 'C:', 'systemroot': 'C:\\WINDOWS', 'temp': 'C:\\Users\\jaraco\\AppData\\Local\\Temp', 'tmp': 'C:\\Users\\jaraco\\AppData\\Local\\Temp', 'userdomain': 'AKUMA', 'userdomain_roamingprofile': 'AKUMA', 'username': 'jaraco', 'userprofile': 'C:\\Users\\jaraco', 'windir': 'C:\\WINDOWS', 'lang': 'en_US.UTF-8', 'term': 'xterm-256color', 'colorterm': 'truecolor', 'term_program': 'Hyper', 'term_program_version': '3.0.2', '_path': 'C:\\Windows\\py.exe', 'pwd': 'C:\\Users\\jaraco', 'path': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.25.28610\\bin\\HostX64\\x64;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\MSBuild\\Current\\bin\\Roslyn;C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.18362.0\\x64;C:\\Program Files (x86)\\Windows Kits\\10\\bin\\x64;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\\\MSBuild\\Current\\Bin;C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\IDE\\;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\Tools\\;C:\\Program Files\\Python38\\Scripts\\;C:\\Program Files\\Python38\\;C:\\Python37\\Scripts\\;C:\\Python37\\;C:\\Python27\\;C:\\Python27\\Scripts;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files\\dotnet\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\Git\\cmd;C:\\Users\\jaraco\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\jaraco\\AppData\\Local\\hyper\\app-3.0.2\\resources\\bin;C:\\Users\\jaraco\\.local\\bin;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin;C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\Ninja', 'loaded_rc_files': 'False,False,True', 'xonshrc': 'C:\\ProgramData\\xonsh\\xonshrc;C:\\Users\\jaraco\\.config\\xonsh\\rc.xsh;C:\\Users\\jaraco/.xonshrc', 'prompt': '$P$G', 'shell_type': 'prompt_toolkit', 'xonsh_login': '1', 'xonsh_interactive': 'True', 'pythonpath': 'C:\\Users\\jaraco\\AppData\\Local\\Temp\\pip-run-wmxdfn5n', '__dotnet_add_64bit': '1', '__dotnet_preferred_bitness': '64', '__vscmd_preinit_path': 'C:\\Program Files\\Python38\\Scripts\\;C:\\Program Files\\Python38\\;C:\\Python37\\Scripts\\;C:\\Python37\\;C:\\Python27\\;C:\\Python27\\Scripts;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files\\dotnet\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\Git\\cmd;C:\\Users\\jaraco\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\jaraco\\AppData\\Local\\hyper\\app-3.0.2\\resources\\bin;C:\\Users\\jaraco\\.local\\bin', 'py_vcruntime_redist': 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build\\..\\..\\redist\\MSVC\\14.25.28508\\onecore\\x64\\Microsoft.VC142.CRT\\vcruntime140.dll'}
jaraco commented 4 years ago

So although I was able to replicate the error, I'm not sure invoking msvc9* is relevant on later Pythons, such as 3.8... and best I can tell, that module isn't relevant on Python 3.5 or later.

Can someone help with a minimal reproducer - a project that fails to build with MSVC 14.24 or 14.25 installed?

Huy-Ngo commented 4 years ago

I cannot reproduce this error with Python 3.7

Source: UNKNOWN-0.0.0.zip

Build log:

D:\Sproject\UNKNOWN-0.0.0>py setup.py build
running build
running build_ext
building 'spam' extension
creating build
creating build\temp.win-amd64-3.7
creating build\temp.win-amd64-3.7\Release
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.25.28610\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\Admin\AppData\Local\Programs\Python\Python37\include -IC:\Users\Admin\AppData\Local\Programs\Python\Python37\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.25.28610\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tcspam.c /Fobuild\temp.win-amd64-3.7\Release\spam.obj
spam.c
creating D:\Sproject\UNKNOWN-0.0.0\build\lib.win-amd64-3.7
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.25.28610\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\Admin\AppData\Local\Programs\Python\Python37\libs /LIBPATH:C:\Users\Admin\AppData\Local\Programs\Python\Python37\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.25.28610\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64" /EXPORT:PyInit_spam build\temp.win-amd64-3.7\Release\spam.obj /OUT:build\lib.win-amd64-3.7\spam.cp37-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.7\Release\spam.cp37-win_amd64.lib
   Creating library build\temp.win-amd64-3.7\Release\spam.cp37-win_amd64.lib and object build\temp.win-amd64-3.7\Release\spam.cp37-win_amd64.exp
Generating code
Finished generating code
jpfeuffer commented 4 years ago

Not sure if this is related, but the nightly builds of our cython module stopped working after GitHub Actions updated from MSVC 14.25.28610 to MSVC 14.26.28801. Setuptools reports the error above. Interestingly it works for Python 3.6 but not for 3.7 (3.8 was not tested due to auto-cancellation).

illume commented 4 years ago

Getting quite a lot of people running into this error the past months.

The magic checkbox seems to be: "[ ] MSVC v140 - VS 2015 C++ build tools (v14.00)"

marekr commented 4 years ago

Fresh install on a build VM. I get the same issue where msvc9_query_vcvarsall complains it cannot find the 2019 build tools in that /14.2/ folder which of course doesn't exist. Meanwhile msvc._msvc14_find_vcvarsall('amd64') can find the 2019 build tools in the correct folder

ikovrigin commented 3 years ago

Was able to fix this issue after some debugging by adding to registry: Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.2\Setup\VC] "productdir"="C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build"

pllim commented 1 year ago

I ran into a similar problem on Python 3.10 still in 2023. None of the suggestions above worked. I did have to fix two things unrelated to Python itself to get my stuff to build with MSVC. While the path discovery worked, it was crashing in cmd call. So this was that I did:

  1. Fix my broken Windows 10 Command Prompt (condabin/conda_hook.bat somehow got itself into its AutoRun and crashing it): https://stackoverflow.com/questions/59480657/cmd-exe-throws-error-was-unexpected-at-this-time
  2. Fire up the Command Prompt (with admin privilege, just in case), run conda init cmd.exe again. Then restarted it, changed to the conda env I wanted, changed to the source checkout using cd /d <dir_name>, did a git clean -xdf to be sure, then I finally could build using pip install -e ..

My real dev setup using VS Code + Git Bash did not work before the fix above, but now seems to work (for now).