search

wafo-project / pywafo

Wave Analysis for Fatigue and Oceanography
http://www.maths.lth.se/matstat/wafo/
Other
94 stars 42 forks source link

pywafo can not be compiled on windows #26

Closed eelcovv closed 8 years ago

eelcovv commented 8 years ago

I have used the latest version of pywafo. I need to have the compiled version since I need the cylce_astm routine. In order to compile the libary I have run

python setup.py bdist_wheel -d build

I have both the mingw compiler and the microsoft visual studio C++ compiler installed, but I can not find how to choose which version should be used and even can not see which version is being used.

I get an error message related to not be able to read the python27.dll

I have included the total outpyt to the log.txt file.

Key message I think is the first warning saying: Building import library (arch=AMD64): "C:\Apps\Anaconda\Anaconda250\libs\libpython27.a" (from C:\Apps\Anaconda\Anaconda250\python27.dll) objdump.exe: C:\Apps\Anaconda\Anaconda250\python27.dll: File format not recognized Traceback (most recent call last): File "setup.py", line 83, in setup_package_pyscaffold()

etc. File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 276, in generate_def raise ValueError("Symbol table not found") ValueError: Symbol table not found

so somehow the python27.dll is not recognised, which I though was related to the mingw compiler.

The question is: how can I fix this or how can I ensure that the CL compiler (msvC++) is used ?

Hopefully you can help me!

Regards Eelco

log.txt

pbrod commented 8 years ago

The mingw toolchain to compile the fortran extensions in pywafo on windows is broken at the moment. However, it is possible to compile only the c-extensions with a microsoft compiler.

To make sure you use the microsoft compiler you can try this: Make a "distutils.cfg" file in "C:\python\Anaconda2\lib\distutils\" and add the following 2 lines to it:

[build] compiler=msvc

You can then try to run the following in a dos-prompt: python setup.py bdist_wheel -d build

If that does not work you can go to "c:\wafo\source\c_library\" folder and run the following in a dos-prompt:

python build_all.py

and the manually copy the c_library.pyd file to the "c:\wafo\" folder.

eelcovv commented 8 years ago

Hi Per,

Thank you very much for your reply. I have the feeling that I am getting closer, but I still get an error.

I did what you told me (adding the distutils.cfg with compiler=msvc (btw: the compiler runs with the command cl.exe, should msvc maybe be cl?)

Anyway, the first command does not work

The second one with python build_all.py give the log as in the log.txt file

Errors start to be raised from

Building import library (arch=AMD64): "C:\Apps\Anaconda\Anaconda250\libs\libpython27.a" (from C:\Apps\Anaconda\Anaconda250\python27.dll) objdump.exe: C:\Apps\Anaconda\Anaconda250\python27.dll: File format not recognized Traceback (most recent call last): File "C:\Apps\Anaconda\Anaconda250\Scripts\f2py.py", line 28, in main()

I read somewhere on a mingw side that the libpython27.a should be generated from the python27.dll file. Does that make sense to you ?

Would you have a clue to fix this ?

Any hints appreciated!

Regards Eelco

2016-06-24 14:57 GMT+02:00 Per A. Brodtkorb notifications@github.com:

The mingw toolchain to compile the fortran extensions in pywafo on windows is broken at the moment. However, it is possible to compile only the c-extensions with a microsoft compiler.

To make sure you use the microsoft compiler you can try this: Make a "distutils.cfg" file in "C:\python\Anaconda2\lib\distutils\" and add the following 2 lines to it:

[build] compiler=msvc

You can then try to run the following in a dos-prompt: python setup.py bdist_wheel -d build

If that does not work you can go to "c:\wafo\source\c_library\" folder and run the following in a dos-prompt:

python build_all.py

and the manually copy the c_library.pyd file to the "c:\wafo\" folder.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wafo-project/pywafo/issues/26#issuecomment-228338476, or mute the thread https://github.com/notifications/unsubscribe/ACtgoOtc5ZUNTvx3OLMbYPylBsVgd8uxks5qO9QtgaJpZM4I9uQd .

running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler options running config_fc unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options running build_src build_src building extension "c_library" sources creating c:\users\eelcovv\appdata\local\temp\tmpgf13m3\src.win-amd64-2.7 f2py options: [] f2py: c_library.pyf Reading fortran codes... Reading file 'c_library.pyf' (format:free) Line #6 in c_library.pyf:" intent(c) " All arguments will have attribute intent(c) Line #15 in c_library.pyf:" intent(c) " All arguments will have attribute intent(c) Line #26 in c_library.pyf:" intent(c) " All arguments will have attribute intent(c) Line #37 in c_library.pyf:" intent(c) " All arguments will have attribute intent(c) Line #53 in c_library.pyf:" intent(c) " All arguments will have attribute intent(c) Line #71 in c_library.pyf:" intent(c) " All arguments will have attribute intent(c) Post-processing... Block: c_library Block: findrfc Block: findcross {'attrspec': ['intent(c)']} In: c_library.pyf:c_library:unknown_interface:disufq vars2fortran: No typespec for argument "m". {'attrspec': ['intent(c)']} In: c_library.pyf:c_library:unknown_interface:disufq vars2fortran: No typespec for argument "n". Block: disufq {'attrspec': ['intent(c)']} In: c_library.pyf:c_library:unknown_interface:disufq2 vars2fortran: No typespec for argument "m". {'attrspec': ['intent(c)']} In: c_library.pyf:c_library:unknown_interface:disufq2 vars2fortran: No typespec for argument "n". Block: disufq2 Block: findrfc3_astm Block: findrfc5_astm Post-processing (stage 2)... Building modules... Building module "c_library"... Constructing wrapper function "findrfc"... ind,info = findrfc(y1,hmin) Constructing wrapper function "findcross"... ind,info = findcross(y,v) Constructing wrapper function "disufq"... rvec,ivec = disufq(rA,iA,w,kw,h,g,nmin,nmax,m,[n]) Constructing wrapper function "disufq2"... rsvec,isvec,rdvec,idvec = disufq2(rA,iA,w,kw,h,g,nmin,nmax,m,[n]) Constructing wrapper function "findrfc3_astm"... array_out,nout = findrfc3_astm(array_ext) Constructing wrapper function "findrfc5_astm"... array_out,nout = findrfc5_astm(array_ext,array_t) Wrote C/API module "c_library" to file "c:\users\eelcovv\appdata\local\temp\tmpgf13m3\src.win-amd64-2.7\c_librarymodule.c" adding 'c:\users\eelcovv\appdata\local\temp\tmpgf13m3\src.win-amd64-2.7\fortranobject.c' to sources. adding 'c:\users\eelcovv\appdata\local\temp\tmpgf13m3\src.win-amd64-2.7' to include_dirs. copying C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\f2py\src\fortranobject.c -> c:\users\eelcovv\appdata\local\temp\tmpgf13m3\src.win-amd64-2.7 copying C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\f2py\src\fortranobject.h -> c:\users\eelcovv\appdata\local\temp\tmpgf13m3\src.win-amd64-2.7 build_src: building npy-pkg config files running build_ext Looking for python27.dll Building import library (arch=AMD64): "C:\Apps\Anaconda\Anaconda250\libs\libpython27.a" (from C:\Apps\Anaconda\Anaconda250\python27.dll) objdump.exe: C:\Apps\Anaconda\Anaconda250\python27.dll: File format not recognized Traceback (most recent call last): File "C:\Apps\Anaconda\Anaconda250\Scripts\f2py.py", line 28, in main() File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\f2py\f2py2e.py", line 648, in main run_compile() File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\f2py\f2py2e.py", line 633, in run_compile setup(ext_modules=[ext]) File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\core.py", line 169, in setup return old_setup(new_attr) File "C:\Apps\Anaconda\Anaconda250\lib\distutils\core.py", line 151, in setup dist.run_commands() File "C:\Apps\Anaconda\Anaconda250\lib\distutils\dist.py", line 953, in run_commands self.run_command(cmd) File "C:\Apps\Anaconda\Anaconda250\lib\distutils\dist.py", line 972, in run_command cmd_obj.run() File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\command\build.py", line 47, in run old_build.run(self) File "C:\Apps\Anaconda\Anaconda250\lib\distutils\command\build.py", line 127, in run self.run_command(cmd_name) File "C:\Apps\Anaconda\Anaconda250\lib\distutils\cmd.py", line 326, in run_command self.distribution.run_command(command) File "C:\Apps\Anaconda\Anaconda250\lib\distutils\dist.py", line 972, in run_command cmd_obj.run() File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\command\build_ext.py", line 117, in run force=self.force) File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\ccompiler.py", line 597, in new_compiler compiler = klass(None, dry_run, force) File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 93, in init** build_import_library() File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 379, in build_import_library return _build_import_library_amd64() File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 401, in _build_import_library_amd64 generate_def(dll_file, def_file) File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 276, in generate_def raise ValueError("Symbol table not found") ValueError: Symbol table not found C:\Apps\Anaconda\Anaconda250\lib\site-packages\wafo\spectrum\core.py:35: UserWarning: Compile the c_library.pyd again! warnings.warn('Compile the c_library.pyd again!') C:\Apps\Anaconda\Anaconda250\lib\site-packages\wafo\spectrum\core.py:40: UserWarning: Compile the cov2mod.pyd again! warnings.warn('Compile the cov2mod.pyd again!')

found f2py in: C:\Apps\Anaconda\Anaconda250\Scripts\f2py.py

compiling c_codes

pbrod commented 8 years ago

Have you checked these solutions?:

http://stackoverflow.com/questions/23814977/f2py-compile-fails-python27-dll-file-format-not-recognized

http://stackoverflow.com/questions/26140192/microsoft-visual-c-compiler-for-python-2-7

pbrod commented 8 years ago

You could also look at this

http://stackoverflow.com/questions/36735436/pythran-file-format-not-recognized-and-unknown-ms-compiler-version-1800

eelcovv commented 8 years ago

Hi Per,

Excellent, I fixed this problem. I was able to compile the c_library I have installed the microsoft visual c compiler for widows (link above). Also I had already installed mingw, which I think is needed for gfortran

Then I did the following command :

python setup.py build --compiler=msvc

This runs but end with an error message 

compiling C sources
C:\Users\eelcovv\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ibuild\src.win-amd64-2.7 -IC:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\core\include -IC:\Apps\Anaconda\Anaconda250\include -IC:\Apps\Anaconda\Anaconda250\PC /Tcbuild\src.win-amd64-2.7\wafo\source\mvn\mvnmodule.c /Fobuild\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvnmodule.obj
Found executable C:\Users\eelcovv\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe
C:\Users\eelcovv\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ibuild\src.win-amd64-2.7 -IC:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\core\include -IC:\Apps\Anaconda\Anaconda250\include -IC:\Apps\Anaconda\Anaconda250\PC /Tcbuild\src.win-amd64-2.7\fortranobject.c /Fobuild\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\fortranobject.obj
compiling Fortran sources
Fortran f77 compiler: C:\MinGW\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -O0
Fortran f90 compiler: C:\MinGW\bin\gfortran.exe -Wall -g -fno-second-underscore -O0
Fortran fix compiler: C:\MinGW\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -O0
compile options: '-Ibuild\src.win-amd64-2.7 -IC:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\core\include -IC:\Apps\Anaconda\Anaconda250\include -IC:\Apps\Anaconda\Anaconda250\PC -c'
gfortran.exe:f77: wafo\source\mvn\mvndst.f
gfortran.exe:f77: build\src.win-amd64-2.7\wafo\source\mvn\mvn-f2pywrappers.f
C:\Users\eelcovv\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:c:\mingw\lib\gcc\mingw32\4.8.1 /LIBPATH:C:\Apps\Anaconda\Anaconda250\libs /LIBPATH:C:\Apps\Anaconda\Anaconda250\PCbuild\amd64 /LIBPATH:C:\Apps\Anaconda\Anaconda250\PC\VS9.0\amd64 /LIBPATH:build\temp.win-amd64-2.7 /LIBPATH:C:\Apps\Anaconda\Anaconda250\libs /LIBPATH:C:\Apps\Anaconda\Anaconda250\PCbuild\amd64 /LIBPATH:C:\Apps\Anaconda\Anaconda250\PC\VS9.0\amd64 /LIBPATH:build\temp.win-amd64-2.7 /EXPORT:initmvn build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvnmodule.obj build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\fortranobject.obj build\temp.win-amd64-2.7\Release\wafo\source\mvn\mvndst.o build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvn-f2pywrappers.o /OUT:build\lib.win-amd64-2.7\wafo\mvn.pyd /IMPLIB:build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvn.lib /MANIFESTFILE:build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvn.pyd.manifest
Found executable C:\Users\eelcovv\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\link.exe
build\temp.win-amd64-2.7\Release\wafo\source\mvn\mvndst.o : fatal error LNK1112: module machine type 'X86' conflicts with target machine type 'x64'
error: Command "C:\Users\eelcovv\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:c:\mingw\lib\gcc\mingw32\4.8.1 /LIBPATH:C:\Apps\Anaconda\Anaconda250\libs /LIBPATH:C:\Apps\Anaconda\Anaconda250\PCbuild\amd64 /LIBPATH:C:\Apps\Anaconda\Anaconda250\PC\VS9.0\amd64 /LIBPATH:build\temp.win-amd64-2.7 /LIBPATH:C:\Apps\Anaconda\Anaconda250\libs /LIBPATH:C:\Apps\Anaconda\Anaconda250\PCbuild\amd64 /LIBPATH:C:\Apps\Anaconda\Anaconda250\PC\VS9.0\amd64 /LIBPATH:build\temp.win-amd64-2.7 /EXPORT:initmvn build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvnmodule.obj build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\fortranobject.obj build\temp.win-amd64-2.7\Release\wafo\source\mvn\mvndst.o build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvn-f2pywrappers.o /OUT:build\lib.win-amd64-2.7\wafo\mvn.pyd /IMPLIB:build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvn.lib /MANIFESTFILE:build\temp.win-amd64-2.7\Release\build\src.win-amd64-2.7\wafo\source\mvn\mvn.pyd.manifest" failed with exit status 1112

Anyway, still the c_library.pyd was created. If I run python setup.py install I get the same error message as before:

Looking for python27.dll
Building import library (arch=AMD64): "C:\Apps\Anaconda\Anaconda250\libs\libpython27.a" (from C:\Apps\Anaconda\Anaconda250\python27.dll)
objdump.exe: C:\Apps\Anaconda\Anaconda250\python27.dll: File format not recognized
Traceback (most recent call last):
  File "setup.py", line 83, in <module>
    setup_package_pyscaffold()
  File "setup.py", line 79, in setup_package_pyscaffold
    **config.todict())
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\core.py", line 169, in setup
    return old_setup(**new_attr)
  File "C:\Apps\Anaconda\Anaconda250\lib\distutils\core.py", line 151, in setup
    dist.run_commands()
  File "C:\Apps\Anaconda\Anaconda250\lib\distutils\dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "C:\Apps\Anaconda\Anaconda250\lib\distutils\dist.py", line 972, in run_command
    cmd_obj.run()
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\command\install.py", line 62, in run
    r = self.setuptools_run()
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\command\install.py", line 56, in setuptools_run
    self.do_egg_install()
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\setuptools\command\install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "C:\Apps\Anaconda\Anaconda250\lib\distutils\cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "C:\Apps\Anaconda\Anaconda250\lib\distutils\dist.py", line 972, in run_command
    cmd_obj.run()
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\setuptools\command\bdist_egg.py", line 160, in run
    self.run_command('build_clib')
  File "C:\Apps\Anaconda\Anaconda250\lib\distutils\cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "C:\Apps\Anaconda\Anaconda250\lib\distutils\dist.py", line 972, in run_command
    cmd_obj.run()
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\command\build_clib.py", line 83, in run
    force=self.force)
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\ccompiler.py", line 597, in new_compiler
    compiler = klass(None, dry_run, force)
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 93, in __init__
    build_import_library()
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 379, in build_import_library
    return _build_import_library_amd64()
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 401, in _build_import_library_amd64
    generate_def(dll_file, def_file)
  File "C:\Apps\Anaconda\Anaconda250\lib\site-packages\numpy\distutils\mingw32ccompiler.py", line 276, in generate_def
    raise ValueError("Symbol table not found")
ValueError: Symbol table not found

However, I I just copy the c_library.pyd manualy from the build directory to the wafo directory the file can be found and I can now run the script using the rainflow cycle_astm again!

So altough the setup script is not working out of the box, I could finally fix it, which is a releave

Thanks for the help !

If you want I have an exampe rain_flow_example.py script which I added to the tutorials which is using this cycle_astm algorithm

Regards Eelco rainflow_example.zip

eelcovv commented 7 years ago

For those interested. Using the same trick as above I was able to compile pywafo for python 3.5.3 as well. For this, I had to first update the Microsoft visual C++ compiler to version 15.0, which can be downloaded from here http://landinghub.visualstudio.com/visual-cpp-build-tools

For the fortran compiler I installed MinGW version 5.3.0-3 and used the gfortran.exe coming with this.

Then I ran: python setup.py build --compiler=msvc

This fails at the end with the same message as above, but again, if you copy the c_library.pyd from the build to the wafo directory you should be able to run pywafo including the rainflow algoritm. Attached an updated version of the rain_flow example. Regards Eelco rainflow_example.zip