search

storpipfugl / pykdtree

Fast kd-tree implementation in Python
GNU Lesser General Public License v3.0
215 stars 47 forks source link

Add basic Windows support #11

Closed djhoese closed 8 years ago

djhoese commented 8 years ago

Windows doesn't seem to like concatenating strings inside a tuple, but it works fine on *nix systems. I saw installation problems on appveyor for python 2.7 and 3.4. This should fix this.

djhoese commented 8 years ago 
Installing collected packages: pykdtree
  Running setup.py install for pykdtree: started
    Running setup.py install for pykdtree: finished with status 'error'
    Complete output from command C:\Python34_32\envs\test\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\pip-build-70i5dsd8\\pykdtree\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\appveyor\AppData\Local\Temp\1\pip-tq_ia3in-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build\lib.win32-3.4
    creating build\lib.win32-3.4\pykdtree
    copying pykdtree\test_tree.py -> build\lib.win32-3.4\pykdtree
    copying pykdtree\__init__.py -> build\lib.win32-3.4\pykdtree
    running build_ext
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\appveyor\AppData\Local\Temp\1\pip-build-70i5dsd8\pykdtree\setup.py", line 83, in <module>
        'Topic :: Scientific/Engineering'
      File "C:\Python34_32\envs\test\lib\distutils\core.py", line 148, in setup
        dist.run_commands()
      File "C:\Python34_32\envs\test\lib\distutils\dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "C:\Python34_32\envs\test\lib\distutils\dist.py", line 974, in run_command
        cmd_obj.run()
      File "C:\Python34_32\envs\test\lib\site-packages\setuptools-23.0.0-py3.4.egg\setuptools\command\install.py", line 61, in run
      File "C:\Python34_32\envs\test\lib\distutils\command\install.py", line 554, in run
        self.run_command('build')
      File "C:\Python34_32\envs\test\lib\distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "C:\Python34_32\envs\test\lib\distutils\dist.py", line 974, in run_command
        cmd_obj.run()
      File "C:\Python34_32\envs\test\lib\distutils\command\build.py", line 126, in run
        self.run_command(cmd_name)
      File "C:\Python34_32\envs\test\lib\distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "C:\Python34_32\envs\test\lib\distutils\dist.py", line 974, in run_command
        cmd_obj.run()
      File "C:\Python34_32\envs\test\lib\site-packages\setuptools-23.0.0-py3.4.egg\setuptools\command\build_ext.py", line 66, in run
      File "C:\Python34_32\envs\test\lib\distutils\command\build_ext.py", line 348, in run
        self.build_extensions()
      File "C:\Users\appveyor\AppData\Local\Temp\1\pip-build-70i5dsd8\pykdtree\setup.py", line 44, in build_extensions
        % comp)
    TypeError: unsupported operand type(s) for %: 'tuple' and 'str'
djhoese commented 8 years ago

Ah just noticed #2, so I'm guessing this doesn't really fix much at all.

djhoese commented 8 years ago

I got pykdtree to work for MSVC on my Windows 7 virtual machine running on python 3.4. I didn't have to change the functionality at all but I did have to move variable declarations to the top of the function...which isn't as pretty as it used to be. @storpipfugl if you have another way of solving this let me know.

FYI here are the current build warnings:

[pytroll_test] C:\Users\User\repos2\git\pykdtree>python setup.py develop
running develop
running egg_info
writing top-level names to pykdtree.egg-info\top_level.txt
writing dependency_links to pykdtree.egg-info\dependency_links.txt
writing pykdtree.egg-info\PKG-INFO
writing requirements to pykdtree.egg-info\requires.txt
reading manifest file 'pykdtree.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pykdtree.egg-info\SOURCES.txt'
running build_ext
building 'pykdtree.kdtree' extension
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Anaconda3\envs\pytroll_test\lib\site-packages\numpy\core\include -IC:\Anaconda3\envs
ytroll_test\include -IC:\Anaconda3\envs\pytroll_test\include /Tcpykdtree/kdtree.c /Fobuild\temp.win-amd64-3.4\Release\pykdtree/kdtree.obj
kdtree.c
c:\anaconda3\envs\pytroll_test\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_
API_VERSION
pykdtree/kdtree.c(2058) : warning C4244: '=' : conversion from 'npy_intp' to 'long', possible loss of data
pykdtree/kdtree.c(2149) : warning C4244: '=' : conversion from 'npy_intp' to 'uint32_t', possible loss of data
pykdtree/kdtree.c(2772) : warning C4244: 'function' : conversion from 'double' to 'float', possible loss of data
pykdtree/kdtree.c(4609) : warning C4244: 'function' : conversion from '__int64' to 'long', possible loss of data
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Anaconda3\envs\pytroll_test\lib\site-packages\numpy\core\include -IC:\Anaconda3\envs
ytroll_test\include -IC:\Anaconda3\envs\pytroll_test\include /Tcpykdtree/_kdtree_core.c /Fobuild\temp.win-amd64-3.4\Release\pykdtree/_kdtree_core.obj
_kdtree_core.c
pykdtree/_kdtree_core.c(708) : warning C4056: overflow in floating-point constant arithmetic
c:\users\user\repos2\git\pykdtree\pykdtree\_kdtree_core.c(708) : warning C4756: overflow in constant arithmetic
creating build\lib.win-amd64-3.4
creating build\lib.win-amd64-3.4\pykdtree
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Anaconda3\envs\pytroll_test\libs /LIBPATH:C:\Anaconda3\envs\pytroll_test\PCbuild\am
4 /EXPORT:PyInit_kdtree build\temp.win-amd64-3.4\Release\pykdtree/kdtree.obj build\temp.win-amd64-3.4\Release\pykdtree/_kdtree_core.obj /OUT:build\lib.win-amd64-3.4\pykdtree\kdtree.pyd /IMPLIB:build
emp.win-amd64-3.4\Release\pykdtree\kdtree.lib /MANIFESTFILE:build\temp.win-amd64-3.4\Release\pykdtree\kdtree.pyd.manifest
kdtree.obj : warning LNK4197: export 'PyInit_kdtree' specified multiple times; using first specification
   Creating library build\temp.win-amd64-3.4\Release\pykdtree\kdtree.lib and object build\temp.win-amd64-3.4\Release\pykdtree\kdtree.exp
copying build\lib.win-amd64-3.4\pykdtree\kdtree.pyd -> pykdtree
Creating c:\anaconda3\envs\pytroll_test\lib\site-packages\pykdtree.egg-link (link to .)

And I currently only get one failure when running tests in pytroll/pyresample:

======================================================================
ERROR: test_dtype (pyresample.test.test_kd_tree.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\users\user\repos2\git\pyresample\pyresample\test\test_kd_tree.py", line 812, in test_dtype
    50000, neighbours=1, segments=1)
  File "c:\users\user\repos2\git\pyresample\pyresample\kd_tree.py", line 380, in get_neighbour_info
    nprocs=nprocs)
  File "c:\users\user\repos2\git\pyresample\pyresample\kd_tree.py", line 560, in _query_resample_kdtree
    distance_upper_bound=radius_of_influence)
  File "kdtree.pyx", line 168, in pykdtree.kdtree.KDTree.query (pykdtree/kdtree.c:2137)
TypeError: Type mismatch. query points must be of type float32 when data points are of type float32

----------------------------------------------------------------------
Ran 114 tests in 83.456s

FAILED (errors=1)

Major plus is that pykdtree gets the pyresample tests done in 83 seconds where scipy takes about 25-30 minutes.

djhoese commented 8 years ago

Sorry, had to force a push. This is the second time I assumed pre-master was the "in progress" branch and based my commits off of that. This PR is now based off of master, but is in "pre-master".

Also, now all the pyresample tests pass.

djhoese commented 8 years ago

With these latest commits pykdtree can run on Windows 32-bit and 64-bit machines with python 2.7, 3.4, and 3.5. This is automatically tested using appveyor (just started using this for pyresample too). The results for my repository and branch are here: https://ci.appveyor.com/project/davidh-ssec/pykdtree

If you guys are OK with this I think it is ready to be merged.

storpipfugl commented 8 years ago

Nice work. I think the C89 style declaration regression is well worth getting it to run on windows

djhoese commented 8 years ago

Thanks. I should note that appveyor doesn't support OpenMP so that is not tested, but on my local VM (Windows 7 Python 3.4) it seemed to work fine. Since there is the environment variable to turn openmp off I don't think it should be a problem.

djhoese commented 8 years ago

@storpipfugl do you think this is ready for a new release on PyPI?

storpipfugl commented 8 years ago

@davidh-ssec definitely. I'll try to get it out this weekend