bibi21000
commented
6 years ago Hi Windows give me headache ... It seems that you MUST use the same compiler used to compile python itself. Look at https://wiki.python.org/moin/WindowsCompilers
Closed dude0001 closed 5 years ago
bibi21000
commented
6 years ago Hi Windows give me headache ... It seems that you MUST use the same compiler used to compile python itself. Look at https://wiki.python.org/moin/WindowsCompilers
dude0001
commented
6 years ago Thank you for the reply. I am using Python 3.7. I did read through that article you linked, but 3.7 isn't listed. I did some googling and it looks like 3.7 still uses C++ 14.0 which is what I am using. You can see in my build log the link.exe line is looking in the v14 directory (I think)
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\link.exe' failed with exit status 1120
Is there something in the build log that is telling you it is using the wrong compiler? Any other thoughts? What would this error mean on a non-windows environment? I don't necessarily think this is Windows related. I think something isn't getting placed in the correct folder, etc as it looks like the compiler isn't able to find the OpenZWave namespaced dependencies.
bibi21000
commented
6 years ago Compilation and linking are ok on Ubuntu and debian (look at travis-ci and circleci jobs) For windows (look at appveyor), the command python setup.py install --flavor=dev and the tests are :
When first testing build for Windows, I've made a fresh VM for windows 10. Everything works fine for python 2.7 / 32 bits. After I try to tests python 2.7 / 64 bits. It brokes all. So I try to remove the 64 bits versions of python and libs and reinstall the 32 bits one ... but it fails too !!!
What can I say ? ... Try on a fresh installation of Windows ... and the most important thing, only install the libraries you really need. That's the Windows world !!!
As said Windows developpment always give me headache ... That's why I leave it 20 years ago.
Maybe, you can try to build openzwave by hand (look at documentation), it can help you.
Sorry, I can not help you anymore
dude0001
commented
6 years ago This is very helpful. I'm trying to learn python while playing with my home automation solution. Thank you for the information, this should help in my learning and troubleshooting the issue. I'll start trying to dig in to this more.
One last question, what documentation for openzwave are you referring to exactly? I've followed a few different sets of instructions. I have been able to build openzwave 64 bit on my machine. It just fails with the setup script in the python-openzwave project. So like I said above, I think something just isn't getting place in the right location or wrong version of a tool being used. The other thing I am looking at is the project file this builds for VS 2017 is still very different from the one that is created when you let VS update it automatically. There are different files that aren't even there and some different build commands it runs. However, it does build. Those things could very well be related though.
dude0001
commented
6 years ago I see what is happening and I think I'm close to a resolution. On Python x64, the scripts are calling devenv.exe /upgrade after some custom upgrade logic that edits the project and solution files. The devenv.exe /upgrade is clobbering the custom stuff. I don't know Python but I'm trying to work it out so the custom stuff is called after devenv.exe /upgrade and removing some of the unneeded custom stuff that devenv.exe /upgrade already does for you. I've got it down to 3 unresolved externals now instead of the 30+ errors I was getting.
One thing I am not clear about is which .py scripts should be called when trying to install from Python 3.6/3.7 x64 on Windows. The appveyor script and the INSTALL_WIN.rst instructions.
I know you have moved on from Windows, but I think I am close to getting this working for x64 and I would love to help. Can you explain the intention or difference between the two or is one just out of date?
appveyor says
build_script:
# Build the compiled extension
- "python pyozw_win.py"
- "python setup.py install --flavor=dev"INSTALL_WIN.rst says nothing about pyozw_win.py and just says
python setup.py install --flavor=dev
Also, appveyor said there is a dependency "nose" that isn't mentioned in INSTALL_WIN.rst
bibi21000
commented
6 years ago pyozw_win.py is a module called by pyozw_setup.py wich is a module called by setup.py. It regroups all needed stuff needed for windows. My experience in windows developmet says to me that it impossible to put all the windows stuff in pyozw_setup.py (like for all others os) This module had a main part ... which allows it to be called directly (look at python doc for information on it). The normal process is python setup.py install python pyozw_win.py is for testing nose is a module used for automatics tests
dude0001
commented
6 years ago Thank you for more info. So in a normal "production" environment you should only run python setup.py install --flavor=dev?
I can see the need as a developer to step in to pyozw_win.py "main" for testing. But why is this called in the appveyor script before python setup.py install --flavor=dev Why don't we just call setup.py there too?
The reason I am hung up on this is, it will influence how I solve the current problem of the call to devenv.exe /upgrade stomping on the project file changes made in the pyozw_win.py update_vs_project. I understand how pyozw_win.py is used to setup the context for windows, get a path to the project, setup the 2017 project path, etc. But why are we doing any upgrade logic there in update_vs_project. I think that is the whole problem.
The issue is depending on if you are executing this from appveyor or doing it in a production environment using just setup.py, it will do different things because some of the code is running twice in appveyor (update_vs_project).
What we really need to do to fix this in both appveyor and a normal production deployment is to do ALL the upgrade logic once and only once. This means do the devenv.exe /upgrade first and then modified version of the pyozw_win.py update_vs_project immediately after. pyozw_win.py update_vs_project shouldn't be called when setting up the windows context.
This is in fact what I am trying to do, but I cannot figure out a straight forward way to do that so it works in both appveyor with the two steps AND a production deployment just calling setup.py.
bibi21000
commented
6 years ago The final goal is to install it using pip (which call python setup.py install --flavor=the good flavor) Appveyor is a testing tool for developpers ... so why not call pyozw_win.py in it ? It can help as we can be more verbose calling it directly Give an atomic PR with your changes (not a one with hundreds of changes, if it breaks something for other os, I won't accept it) If needed, move some code from pyozw_setup into pyozw_win (but not in the other way)
kdschlosser
commented
6 years ago I wanted to give a heads up on this. the vast majority of the windows code can be removed.
if you modify the vs2010 solution so it is compatible with VS2017 locate vs2017 in the reg grab the VC directory run vcvarsall for the specific architecture and capture the data return by set. apply the differences to os.environ. apply the toolkit and SDK to the VS solution. and use msbuild to build it.
if you add the environment variables DISTUTILS_USE_SDK= to os.environ and MSSDK=the sdk path as well. it will build properly for all python versions any architecture.
So basically in order to simplify the build process for windows make it a requirement to have Visual Studio 2017 with all of the Visual C components added and Windows SDK 8.1 or above.
Between Microsoft and Python builds it is almost impossible to try to build openzwave using what is on a users system. because Microsoft has no consistent mechanism in place for making this easy the easiest solution is to tell the user they have to have these things in order for it to build.
I do know of the possibility of having problems if the version of python was compiled with a different compiler as openzwave. I have found no such issues. it work without any problems. with the above setup.
You do have to make sure that openzwave is built using the same compiler that Cython uses to build the pyd file. that is what the DISTUTILS_USE_SDK and MSSDK env variables do. MSSDK needs to be set to the same path as WINDOWSSDKVERBINPATH
The biggest problem is with python 2.7, 3.1, 3.2 if you want to build using the compiler that python is built in. because trying to locate the proper SDK's for the MSVC compiler 9.0 as well as the .net toolchain is difficult. not to mention the issues with installing the SDK's bumping heads with some of the VC Redist's and .Net.
the above works and it works without error every single time. I suggest making it a requirement.
I do have a test set of scripts to build libopenzwave if it is wanted for example. this set of scripts does properly parse all of the different build variables like the SDK and toolsets the problem is locating the components necessary to build it successfully. if the Visual C compiler for python gets used. (which probably would be the case most of the time on 2.7, 3.1, and 3.2) it will always fail. because of an error in one of the included files. it doesn't properly map bool when compiling C code. and the Encryption bits in openzwave are written in C
I even went to the extent of creating an Extension and adding all of the necessary compiler args macros include paths and files so distutils can build it. It still fails if the python C compiler is used. I did manage to find a trial version of VS 2008 on Microsoft's website. I tried to build the solution included with openzwave for VS2008. it fails.
Trying to build on windows and I am stuck. This is the output of the build. I see it is failing in the linking step, but I don't know how to troubleshoot from here.
I can see libopenzwave.cpp is being built and copied up to python-openzwave\src-lib\libopenzwave. But something appears to be wrong with it I guess.
Let me know if you want to look at anything in my environment or if you need more details.