src-d / kmcuda

Large scale K-means and K-nn implementation on NVIDIA GPU / CUDA
Other
791 stars 145 forks source link

Install instruction gives an error #11

Closed olalonde closed 7 years ago

olalonde commented 7 years ago

On Mac with Python 3, the pip install command doesn't work:

$ pip install git+https://github.com/src-d/kmcuda.git
Collecting git+https://github.com/src-d/kmcuda.git
  Cloning https://github.com/src-d/kmcuda.git to /private/var/folders/67/wj749ptn3jlb4_nlf9nphyy00000gn/T/pip-_77ac4rm-build
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tokenize.py", line 452, in open
        buffer = _builtin_open(filename, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/67/wj749ptn3jlb4_nlf9nphyy00000gn/T/pip-_77ac4rm-build/setup.py'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/67/wj749ptn3jlb4_nlf9nphyy00000gn/T/pip-_77ac4rm-build/
olalonde commented 7 years ago

I'm also getting a build error on mac when trying to compile:

~/code/kmcuda/src$ CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ LDFLAGS=-L/usr/local/opt/llvm/lib/ cmake -DCMAKE_BUILD_TYPE=Release .                                    develop
-- The C compiler identification is Clang 3.8.0
-- The CXX compiler identification is Clang 3.8.0
-- Check for working C compiler: /usr/local/opt/llvm/bin/clang
-- Check for working C compiler: /usr/local/opt/llvm/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/local/opt/llvm/bin/clang++
-- Check for working CXX compiler: /usr/local/opt/llvm/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Try OpenMP C flag = [-fopenmp=libomp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [ ]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [/openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-Qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-xopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [+Oopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-qsmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-mp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-fopenmp=libomp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [ ]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [/openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-Qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-xopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [+Oopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-qsmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-mp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
CMake Error at /usr/local/Cellar/cmake/3.7.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
  Could NOT find OpenMP (missing: OpenMP_C_FLAGS OpenMP_CXX_FLAGS)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.7.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/Cellar/cmake/3.7.2/share/cmake/Modules/FindOpenMP.cmake:316 (find_package_handle_standard_args)
  CMakeLists.txt:4 (find_package)

-- Configuring incomplete, errors occurred!
See also "/Users/olalonde/code/kmcuda/src/CMakeFiles/CMakeOutput.log".
See also "/Users/olalonde/code/kmcuda/src/CMakeFiles/CMakeError.log".
vmarkovtsev commented 7 years ago

This is weird because it works In Travis. The first error means that there is no setup.py in the root dir. Indeed, it is not after the latest refactoring, at the moment you have to change to src. The second error violates the condition stated in the Readme. Your clang does not support openmp. This has nothing to do with kmcuda, probably Homebrew screwed it up lately. Try to build a sample openmp prog using it.

olalonde commented 7 years ago

Ah, I see. I will try to find a workaround. By the way, if the travis build works for OS X, it seems like it wouldn't be that hard to distribute the compiled module for macos as well?

olalonde commented 7 years ago
~/code/kmcuda/src$ brew install llvm --with-clang                                                                                                                                         develop
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/science).
No changes to formulae.

Warning: llvm is a keg-only and another version is linked to opt.
Use `brew install --force` if you want to install this version
(venv) ~/code/kmcuda/src$ brew install llvm --with-clang --force                                                                                                                                 develop
Warning: llvm: this formula has no --with-clang option so it will be ignored!
==> Downloading https://homebrew.bintray.com/bottles/llvm-4.0.0.sierra.bottle.tar.gz

Looks like llvm no longer has --with-clang option. After re-installing llvm, I'm getting a new error:

$ CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ LDFLAGS=-L/usr/local/opt/llvm/lib/ cmake -DCMAKE_BUILD_TYPE=Release .                             develop
-- Try OpenMP C flag = [-fopenmp=libomp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-fopenmp=libomp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
CMake Error at /usr/local/Cellar/cmake/3.7.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
  Could NOT find OpenMP (missing: OpenMP_C_FLAGS OpenMP_CXX_FLAGS)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.7.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/Cellar/cmake/3.7.2/share/cmake/Modules/FindOpenMP.cmake:316 (find_package_handle_standard_args)
  CMakeLists.txt:4 (find_package)

-- Configuring incomplete, errors occurred!
See also "/Users/olalonde/code/kmcuda/src/CMakeFiles/CMakeOutput.log".
See also "/Users/olalonde/code/kmcuda/src/CMakeFiles/CMakeError.log".

I tried disabling the openmp check in cmakelist:

--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -1,7 +1,7 @@
 cmake_minimum_required(VERSION 3.2)
 project(KMCUDA)
 set(CMAKE_MODULE_PATH ${CMAKE_HOME_DIRECTORY}/cmake)
-find_package(OpenMP REQUIRED)
+# find_package(OpenMP REQUIRED)
 if (APPLE AND NOT CUDA_HOST_COMPILER)
   # https://gitlab.kitware.com/cmake/cmake/issues/13674
   message(WARNING "CUDA_HOST_COMPILER is set to the default /usr/bin/clang; CUDA 8.0 requires 7.3 or older.")

And the command now finishes, but I'm later getting another problem:

~/code/kmcuda/src$ CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ LDFLAGS=-L/usr/local/opt/llvm/lib/ cmake -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles" .
-- Found python3-config: /usr/local/bin/python3-config
-- Discovered Python 3.x prefix: /usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6
-- Could NOT find R (missing:  R_EXECUTABLE R_INCLUDE_DIR R_LIBRARY)
-- Applied CUDA OpenMP macOS workaround
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/olalonde/code/kmcuda/src
~/code/kmcuda/src$ make                                                                                                                                                                
[ 16%] Building NVCC (Device) object CMakeFiles/KMCUDA.dir/KMCUDA_generated_transpose.cu.o
nvcc fatal   : The version ('80100') of the host compiler ('Apple clang') is not supported
CMake Error at KMCUDA_generated_transpose.cu.o.Release.cmake:218 (message):
  Error generating
  /Users/olalonde/code/kmcuda/src/CMakeFiles/KMCUDA.dir//./KMCUDA_generated_transpose.cu.o

make[2]: *** [CMakeFiles/KMCUDA.dir/KMCUDA_generated_transpose.cu.o] Error 1
make[1]: *** [CMakeFiles/KMCUDA.dir/all] Error 2
make: *** [all] Error 2
olalonde commented 7 years ago

Ok, so I think my Macbook Pro doesn't even have a CUDA compatible GPU :| Sorry to waste your time.

screen shot 2017-04-09 at 12 05 16
vmarkovtsev commented 7 years ago

@olalonde I fixed the pip install cmdline so that the original issue is resolved. The solution is

pip3 install git+https://github.com/src-d/kmcuda.git#subdirectory=src