search

src-d / kmcuda

Large scale K-means and K-nn implementation on NVIDIA GPU / CUDA
Other
797 stars 146 forks source link

Installation issue with cmake (CUDA_curand_LIBRARY notfound) #93

Closed tsologub closed 5 years ago

tsologub commented 5 years ago

Hi. I am using: cuda - 10.0 cmake - 3.15.2

During installation on Linux with pip install libKMCUDA got the following stacktrace. It seems that cmake can't find cuda libraries. Right? 

Collecting libKMCUDA
  Using cached https://files.pythonhosted.org/packages/2e/2f/07c103a2770b2261a70b6b0a1fb644a7bf561789c162763eee9cfd45d9b1/libKMCUDA-6.2.3.tar.gz
Requirement already satisfied: numpy in /usr/local/lib/python3.5/dist-packages (from libKMCUDA) (1.16.3)
Building wheels for collected packages: libKMCUDA
  Building wheel for libKMCUDA (setup.py) ... error
  ERROR: Complete output from command /usr/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-0956wpof/libKMCUDA/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-yvv7rgpk --python-tag cp35:
  ERROR: running bdist_wheel
  running build
  running build_py
  -- The C compiler identification is GNU 5.4.0
  -- The CXX compiler identification is GNU 5.4.0
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Found OpenMP_C: -fopenmp (found version "4.0")
  -- Found OpenMP_CXX: -fopenmp (found version "4.0")
  -- Found OpenMP: TRUE (found version "4.0")
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3")
  -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so (found suitable version "3.5.2", minimum required is "3")
  CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
  Please set them or make sure they are set and tested correctly in the CMake files:
  CUDA_curand_LIBRARY (ADVANCED)
      linked by target "KMCUDA" in directory /tmp/pip-install-0956wpof/libKMCUDA

  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-0956wpof/libKMCUDA/CMakeFiles/CMakeOutput.log".
  See also "/tmp/pip-install-0956wpof/libKMCUDA/CMakeFiles/CMakeError.log".
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-0956wpof/libKMCUDA/setup.py", line 107, in <module>
      "Programming Language :: Python :: 3.7",
    File "/usr/local/lib/python3.5/dist-packages/setuptools/__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 179, in run
      self.run_command('build')
    File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.5/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-0956wpof/libKMCUDA/setup.py", line 20, in run
      self._build()
    File "/tmp/pip-install-0956wpof/libKMCUDA/setup.py", line 38, in _build
      "."))
    File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '('cmake', '-DCMAKE_BUILD_TYPE=Release', '-DDISABLE_R=y', '-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda', '-DCUDA_ARCH=61', '.')' returned non-zero exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for libKMCUDA
  Running setup.py clean for libKMCUDA
Failed to build libKMCUDA
Installing collected packages: libKMCUDA
  Running setup.py install for libKMCUDA ... error
    ERROR: Complete output from command /usr/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-0956wpof/libKMCUDA/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-23xszfe9/install-record.txt --single-version-externally-managed --compile:
    ERROR: running install
    running build
    running build_py
    -- Found OpenMP_C: -fopenmp
    -- Found OpenMP_CXX: -fopenmp
    -- Found OpenMP: TRUE
    CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
    Please set them or make sure they are set and tested correctly in the CMake files:
    CUDA_curand_LIBRARY (ADVANCED)
        linked by target "KMCUDA" in directory /tmp/pip-install-0956wpof/libKMCUDA

    -- Configuring incomplete, errors occurred!
    See also "/tmp/pip-install-0956wpof/libKMCUDA/CMakeFiles/CMakeOutput.log".
    See also "/tmp/pip-install-0956wpof/libKMCUDA/CMakeFiles/CMakeError.log".
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-0956wpof/libKMCUDA/setup.py", line 107, in <module>
        "Programming Language :: Python :: 3.7",
      File "/usr/local/lib/python3.5/dist-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.5/dist-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.5/distutils/command/install.py", line 583, in run
        self.run_command('build')
      File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.5/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-0956wpof/libKMCUDA/setup.py", line 20, in run
        self._build()
      File "/tmp/pip-install-0956wpof/libKMCUDA/setup.py", line 38, in _build
        "."))
      File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '('cmake', '-DCMAKE_BUILD_TYPE=Release', '-DDISABLE_R=y', '-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda', '-DCUDA_ARCH=61', '.')' returned non-zero exit status 1
    ----------------------------------------
ERROR: Command "/usr/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-0956wpof/libKMCUDA/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-23xszfe9/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-0956wpof/libKMCUDA/
tsologub commented 5 years ago

not relevant anymore. can't reproduce.

ghost commented 3 years ago

I reproduced this issue with a Jetson Xavier NX on Jetpack 4.4 Cuda 10.2 (L4T Base Image R32.4.4).

@vmarkovtsev one simplification to CMakeLists.txt could be to add Cuda as a language:

It is no longer necessary to use this module or call find_package(CUDA) for compiling CUDA code. Instead, list CUDA among the languages named in the top-level call to the project() command, or call the enable_language() command with CUDA. Then one can add CUDA (.cu) sources to programs directly in calls to add_library() and add_executable(). Source: https://cmake.org/cmake/help/latest/module/FindCUDA.html

I don't know for certain, but this also might fix a bug in locating: /usr/local/cuda-10.2/targets/aarch64-linux/lib/stubs/libcurand.so

To resolve this otherwise, I had to add that stubs directory to my path, which is not what CUDA recommends doing (they only recommend adding the cuda bin to your path):

# per CUDA quick start guide (libKMCUDA also needs targets on path to resolve libcurand): https://docs.nvidia.com/cuda/cuda-quick-start-guide/index.html
ENV PATH=/usr/local/cuda-10.2/bin:/usr/local/cuda-10.2/targets/aarch64-linux/lib/stubs${PATH:+:${PATH}}
ENV LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

RUN CUDA_ARCH=$architecture \
    CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.2 \
    python3 -m pip install libKMCUDA