Open rwarren opened 2 years ago
Hi @rwarren thank you for reporting this.
I had a quick look on your reproducer and I think I will leave this issue for the other maintainers since I don't have any familiarity with this part of the code, but I would like to ask, if it is possible to update your reproducer to do a import setuptools
as the first thing before importing any distutils
? (Just to make sure all the setuptools
patches are applied before anyone starts to investigate...)
Thanks, @abravalheri.
I added a simple import setuptools
as follows:
import setuptools # as requested
import distutils.msvccompiler as msvc
mc = msvc.MSVCCompiler()
mc.initialize() # <-- BOOM
No change. This unfortunately throws the same exception.
FWIW, in the event it was something like this (i.e. usage error I can't figure out) I also reported this downstream in the WxPython repo where I first saw the problem before digging deeper: https://github.com/wxWidgets/Phoenix/issues/2165
Over at the downstream location where I encountered this problem (here), @DKMR figured out how to fool distutils
into locating the VS2019 build tools.
If you manually create the the following registry keys (what I'm showing is a .reg
file contents), the mc.initialize()
in the above script works fine.
Windows Registry Editor Version 5.00
; These registry keys are needed to coerce python into detecting VS2019 build tools
; - adjust `ProductDir` paths as necessary (the dir should be where `vcvarsall.bat` is located)
[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.2\Setup\VC]
"ProductDir"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build"
[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.0\Setup\VC]
"ProductDir"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Auxiliary\\Build"
I think this just confirms that Microsoft made some adjustments to where they keep stuff and/or register stuff.
I have the same problems when I install wxPython via pip install -U wxPython
.
Since I installed vs 2019 Community version, so the path I changed as follows. And then I installed wxPython successfully.
Windows Registry Editor Version 5.00
; setuptools-vs-fix.reg
; These registry keys are needed to coerce python into detecting VS2019 build tools
; - adjust `ProductDir` paths as necessary (the dir should be where `vcvarsall.bat` is located)
[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.2\Setup\VC]
"ProductDir"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Auxiliary\\Build"
[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.0\Setup\VC]
"ProductDir"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Auxiliary\\Build"
Updated:
I have both vs 2019 and 2022 installed, after I configured the regedit above, I got some errors around _nanosvg.obj : error LNK2001: unresolved external symbol _PyGen_Send
and I found the vs tools the building process used are mixed with both 2019 and 2022. So I changed the regedit settings to 2022 and tried again.
Windows Registry Editor Version 5.00
; These registry keys are needed to coerce python into detecting VS2022 build tools
; - adjust `ProductDir` paths as necessary (the dir should be where `vcvarsall.bat` is located)
[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.2\Setup\VC]
"ProductDir"="C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Auxiliary\\Build"
[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\14.0\Setup\VC]
"ProductDir"="C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Auxiliary\\Build"
Then the following errors occured.
@liudonghua123 since Python 3.10 has no available wheels for wxPython and you're building the whole package from ground up, I'd suggest you try building the latest snapshot of wxPython and see if that works out.
@DKMR Thanks for the suggestions, I tried to build on the master branch. But after I fix from collections import Mapping
issue on python 3.9+, I still got the following errors, configure.in No such file or directory
primarily.
D:\code\python\Phoenix>pip install .
Processing d:\code\python\phoenix
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "D:\code\python\Phoenix\setup.py", line 27, in <module>
from buildtools.config import Config, msg, opj, runcmd, canGetSOName, getSOName
File "D:\code\python\Phoenix\buildtools\config.py", line 30, in <module>
from attrdict import AttrDict
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\attrdict\__init__.py", line 5, in <module>
from attrdict.mapping import AttrMap
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\attrdict\mapping.py", line 8, in <module>
from attrdict.mixins import MutableAttr
File "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\attrdict\mixins.py", line 5, in <module>
from collections import Mapping, MutableMapping, Sequence
ImportError: cannot import name 'Mapping' from 'collections' (C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\collections\__init__.py)
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
D:\code\python\Phoenix>pip install .
Processing d:\code\python\phoenix
Preparing metadata (setup.py) ... done
Requirement already satisfied: pillow in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from wxPython==4.2.0a1) (9.0.1)
Requirement already satisfied: six in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from wxPython==4.2.0a1) (1.16.0)
Requirement already satisfied: numpy in c:\users\liu.d.h\appdata\local\programs\python\python310\lib\site-packages (from wxPython==4.2.0a1) (1.22.3)
Building wheels for collected packages: wxPython
Building wheel for wxPython (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [39 lines of output]
C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\dist.py:771: UserWarning: Usage of dash-separated 'license-file' will not be supported in future versions. Please use the underscore name 'license_file' instead
warnings.warn(
C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\config\setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
warnings.warn(msg, warning_class)
C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\dist.py:317: DistDeprecationWarning: use_2to3 is ignored.
warnings.warn(f"{attr} is ignored.", DistDeprecationWarning)
running bdist_wheel
running build
Will build using: "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\python.exe"
3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)]
Python's architecture is 64bit
cfg.VERSION: 4.2.0a1
Running command: build
Running command: build_wx
CL.exe: C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\bin\HostX64\x64\cl.exe
Downloading microsoft.web.webview2 1.0.1185.39...
Connection successful...
Data downloaded...
wxWidgets build options: ['--wxpython', '--unicode']
ERROR: failed building wxWidgets
Traceback (most recent call last):
File "D:\code\python\Phoenix\build.py", line 1560, in cmd_build_wx
wxbuild.main(wxDir(), build_options)
File "D:\code\python\Phoenix\buildtools\build_wxwidgets.py", line 174, in main
VERSION = tuple([int(i) for i in getWxRelease().split('.')[:2]])
File "D:\code\python\Phoenix\buildtools\build_wxwidgets.py", line 81, in getWxRelease
with open(os.path.join(wxRoot, "configure.in"), "r") as fid:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\code\\python\\Phoenix\\ext\\wxWidgets\\configure.in'
Finished command: build_wx (0m8.749s)
Finished command: build (0m8.749s)
WARNING: Building this way assumes that all generated files have been
generated already. If that is not the case then use build.py directly
to generate the source and perform the build stage. You can use
--skip-build with the bdist_* or install commands to avoid this
message and the wxWidgets and Phoenix build steps in the future.
"C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\python.exe" -u build.py build
Command '"C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\python.exe" -u build.py build' failed with exit code 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for wxPython
Running setup.py clean for wxPython
Failed to build wxPython
Installing collected packages: wxPython
Running setup.py install for wxPython ... error
error: subprocess-exited-with-error
× Running setup.py install for wxPython did not run successfully.
│ exit code: 1
╰─> [38 lines of output]
C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\dist.py:771: UserWarning: Usage of dash-separated 'license-file' will not be supported in future versions. Please use the underscore name 'license_file' instead
warnings.warn(
C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\config\setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
warnings.warn(msg, warning_class)
C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\dist.py:317: DistDeprecationWarning: use_2to3 is ignored.
warnings.warn(f"{attr} is ignored.", DistDeprecationWarning)
running install
C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
Will build using: "C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\python.exe"
3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)]
Python's architecture is 64bit
cfg.VERSION: 4.2.0a1
Running command: build
Running command: build_wx
CL.exe: C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\bin\HostX64\x64\cl.exe
wxWidgets build options: ['--wxpython', '--unicode']
ERROR: failed building wxWidgets
Traceback (most recent call last):
File "D:\code\python\Phoenix\build.py", line 1560, in cmd_build_wx
wxbuild.main(wxDir(), build_options)
File "D:\code\python\Phoenix\buildtools\build_wxwidgets.py", line 174, in main
VERSION = tuple([int(i) for i in getWxRelease().split('.')[:2]])
File "D:\code\python\Phoenix\buildtools\build_wxwidgets.py", line 81, in getWxRelease
with open(os.path.join(wxRoot, "configure.in"), "r") as fid:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\code\\python\\Phoenix\\ext\\wxWidgets\\configure.in'
Finished command: build_wx (0m2.300s)
Finished command: build (0m2.300s)
WARNING: Building this way assumes that all generated files have been
generated already. If that is not the case then use build.py directly
to generate the source and perform the build stage. You can use
--skip-build with the bdist_* or install commands to avoid this
message and the wxWidgets and Phoenix build steps in the future.
"C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\python.exe" -u build.py build
Command '"C:\Users\Liu.D.H\AppData\Local\Programs\Python\Python310\python.exe" -u build.py build' failed with exit code 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> wxPython
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
D:\code\python\Phoenix>
@liudonghua123 I'm not sure you followed the instructions contained in README.rst file you extracted from the snapshot. TL;DR, install these packages with pip if you don't have them already:
Then you'll need VS Studio 2015 or newer and a copy of MS SDK.
After you have all of the above, build wxPython with python build.py dox etg --nodoc sip build
. Run python build.py install
afterwards to copy wx files to the right place.
import setuptools # as requested
import distutils.msvccompiler as msvc
This is incorrect. The proper module is distutils._msvccompiler
(with an underscore because you're not supposed to access it directly 😉 )
The one you're importing is the ~2.7 implementation, which would have been removed when the new one was added, but I was told to leave it there for compatibility. So yeah, it is totally busted, but the proper one should be fine.
@zooba It's in the API, so it's expected to be correct. That's why it was patching itself.
Perhaps, but the point is that it's not been updated for building in Python 3.5 and later (and now that it's removed from core, those docs are basically irrelevant).
If setuptools decides to update it to work again, that's on them, but my point was that the code people are writing to test things in this thread are all using the wrong class, and the registry keys being suggested aren't ever referenced in the current implementation. So if those people want to prove any points, they need to import a different class.
@liudonghua123 I'm not sure you followed the instructions contained in README.rst file you extracted from the snapshot. TL;DR, install these packages with pip if you don't have them already:
- appdirs
- setuptools
- sip == 6.6.2
- wheel
- twine
- requests
- requests[security]
- cython==0.29.24
- pytest
- pytest-xdist
- pytest-forked
- pytest-timeout
- sphinx==2.2.0 ; python_version >= '3.0'
- Jinja2==2.10
- doc2dash==2.3.0
- beautifulsoup4
- attrdict3
Then you'll need VS Studio 2015 or newer and a copy of MS SDK.
After you have all of the above, build wxPython with
python build.py dox etg --nodoc sip build
. Runpython build.py install
afterwards to copy wx files to the right place.
Thanks, I will try your tips later. :smile:
Command:
python setup.py build_ext --inplace
running build_ext
building 'helloworld' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
The solutions above are not working for me thus far am still investigating... I am running version 3.10 python, windows 11. I have tried anaconda 3.7 and 3.9 still no luck. I have install visual studio Build Tools 2022 and Visual Studio Community 2022 all files which is around 200gb as well as added C:\Program Files (x86)\Microsoft Visual Studio 14.0 and C:\Program Files (x86)\Microsoft Visual Studio 14.0/Vc to path. I have tried https://github.com/abbodi1406/vcredist/releases/tag/v0.62.0
@oddsmatcheraustralia Try running this command and see whether your installs are there:
"C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -prerelease -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64
If they're not showing up, remove the -requires
argument and try again. If they're there now, you need to add the C++ compiler. If they're still not there, they probably need to be repaired. Either way, run "Visual Studio Installer" from Start to do it.
I encountered the same problem at first, but eventually I was able to build it.
In my case, the problem was that vcvarsall.bat
was not installed, but this was solved by specifying the appropriate package in the Build Tools vor Visual Studio 2022 installer.
MSVC v143 - VS 2022 C++ x64/x86 build tools (v14.35-17.5)
does not works for me. it doesn't install vcvarsall.bat
.After installation, vcvarsall.bat
is installed in C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build
.
Here is the configuration file .vsconfig
exported after installation
{
"version": "1.0",
"components": [
"Microsoft.VisualStudio.Component.Roslyn.Compiler",
"Microsoft.Component.MSBuild",
"Microsoft.VisualStudio.Component.CoreBuildTools",
"Microsoft.VisualStudio.Workload.MSBuildTools",
"Microsoft.VisualStudio.Component.VC.Tools.x86.x64",
"Microsoft.VisualStudio.Component.Windows11SDK.22000",
"Microsoft.VisualStudio.Component.VC.14.35.17.5.ATL.Spectre",
"Microsoft.VisualStudio.Component.VC.14.35.17.5.MFC.Spectre"
]
}
setuptools version
setuptools == 62.3.1
Python version
Python 3.10
OS
Win10
Additional environment information
I have tried to make this work on Win10 with various combinations of:
With the latest VS2019 build tools, I'm installing these components (as directed by the compiler wiki):
Description
Main problem: Basic MS builds do not currently work at all on Win10 with currently available VS2019 Build Tools!
The fastest way to see/reproduce this is to try and initialize a
distutils.msvccompiler.MSVCCompiler
with any clean Win10 environment combination and the a VS2019 C++ Build Tools install (with any variety of combinations, as as indicated in the env section).Specifically, this code throws a
DistutilsPlatformError
exception because it can't find Microsoft Visual C++ 14.2:The specific exception thrown, with message, is:
This error seems like it is due to the following things:
C:\Program Files (x86)\Microsoft Visual Studio 14.2\VC\Tools\MSVC
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
MSVC 142 - VS 2019 C++ x64/x86
build tools are installed, it does not install them to the version-dotted dir schema that setuptools is expectingWhen installing build tools today (with the MSVC142 tools selected), the proper location for executables like
cl.exe
is in dirs like this:There are no older MSVC build tools installers available to get a 14.2-based install dir. The URL I am trying to downlad build tools from is https://visualstudio.microsoft.com/vs/older-downloads/, as given by the compiler wiki.
Further details:
Stepping through the
setuptools.msvc
implementation, the following things can be seen:setuptools
does pick up the VS 2019 build tools dir as follows (inSystemInfo.known_vs_paths
):{16.11: 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools'}
2019
dir in the path)setuptools
is looking for is 14.2C:\Program Files (x86)\Microsoft Visual Studio 14.2
For search purposes, I ran into this problem when trying to build WxPython with a simple
pip install wxpython
.Expected behavior
setuptools should work with the currently available VS2019 build tools from Microsoft.
Alternatively, the compiler wiki (or other docs?) should be corrected to point to whatever location has compatible build tools.
How to Reproduce
As described in the description. Essentially:
pip install setuptools -U
to get the latest (just in case)Output