src-d / kmcuda

Large scale K-means and K-nn implementation on NVIDIA GPU / CUDA
Other
807 stars 146 forks source link

Install Issue #71

Closed marsggbo closed 5 years ago

marsggbo commented 5 years ago

I tried to install kmcuda using original pip, but it seems not work.

xinhe@host144:~$ CUDA_ARCH=61 pip3 install libKMCUDA
Collecting libKMCUDA
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/ff/1f/0cf14b8b3e1ec418ab7c4b2fa65c43591a52d27de39b2e976f063554c6a2/libKMCUDA-6.2.2.tar.gz
Collecting numpy (from libKMCUDA)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/35/d5/4f8410ac303e690144f0a0603c4b8fd3b986feb2749c435f7cdbb288f17e/numpy-1.16.2-cp36-cp36m-manylinux1_x86_64.whl (17.3MB)
    100% |████████████████████████████████| 17.3MB 121kB/s
Building wheels for collected packages: libKMCUDA
  Running setup.py bdist_wheel for libKMCUDA ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp65ij8f6ppip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py", line 83, in <module>
      "Programming Language :: Python :: 3.6",
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 129, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 204, in run
      self.run_command('build')
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py", line 19, in run
      self._build()
    File "/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py", line 33, in _build
      "CUDA_TOOLKIT_ROOT_DIR environment variable must be defined")
  __main__.SetupConfigurationError: CUDA_TOOLKIT_ROOT_DIR environment variable must be defined

  ----------------------------------------
  Failed building wheel for libKMCUDA
  Running setup.py clean for libKMCUDA
Failed to build libKMCUDA
Installing collected packages: numpy, libKMCUDA
  Running setup.py install for libKMCUDA ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-wu7mhmqa-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    running install
    running build
    running build_py
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py", line 83, in <module>
        "Programming Language :: Python :: 3.6",
      File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 129, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.6/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py", line 19, in run
        self._build()
      File "/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py", line 33, in _build
        "CUDA_TOOLKIT_ROOT_DIR environment variable must be defined")
    __main__.SetupConfigurationError: CUDA_TOOLKIT_ROOT_DIR environment variable must be defined

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_k65pp9x/libKMCUDA/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-wu7mhmqa-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-_k65pp9x/libKMCUDA/
smujiang commented 5 years ago

you should define your environment variables first. for example, append the following line into your .bashrc CUDA_TOOLKIT_ROOT_DIR=/path_to_your_cuda_toolkit

abhipn commented 5 years ago

I've had a similar error when installing it via pip3. So I've tried to set environment variable for cuda toolkit. export CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda Then again I've tried to install but gave a different error. pip install libKMCUDA

Collecting libKMCUDA
  Using cached https://files.pythonhosted.org/packages/ff/1f/0cf14b8b3e1ec418ab7c4b2fa65c43591a52d27de39b2e976f063554c6a2/libKMCUDA-6.2.2.tar.gz
Collecting numpy (from libKMCUDA)
  Using cached https://files.pythonhosted.org/packages/35/d5/4f8410ac303e690144f0a0603c4b8fd3b986feb2749c435f7cdbb288f17e/numpy-1.16.2-cp36-cp36m-manylinux1_x86_64.whl
Building wheels for collected packages: libKMCUDA
  Running setup.py bdist_wheel for libKMCUDA ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-zrd6twth/libKMCUDA/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpis2862eupip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  CMake Error: The source directory "/tmp/pip-build-zrd6twth/libKMCUDA" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-zrd6twth/libKMCUDA/setup.py", line 83, in <module>
      "Programming Language :: Python :: 3.6",
    File "/home/abhi/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/home/abhi/.local/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 192, in run
      self.run_command('build')
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-zrd6twth/libKMCUDA/setup.py", line 19, in run
      self._build()
    File "/tmp/pip-build-zrd6twth/libKMCUDA/setup.py", line 37, in _build
      "."))
    File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '('cmake', '-DCMAKE_BUILD_TYPE=Release', '-DDISABLE_R=y', '-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda', '-DCUDA_ARCH=61', '.')' returned non-zero exit status 1.

  ----------------------------------------
  Failed building wheel for libKMCUDA
  Running setup.py clean for libKMCUDA
Failed to build libKMCUDA
Installing collected packages: numpy, libKMCUDA
  Running setup.py install for libKMCUDA ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-zrd6twth/libKMCUDA/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-hiebv4_y-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    running install
    running build
    running build_py
    CMake Error: The source directory "/tmp/pip-build-zrd6twth/libKMCUDA" does not appear to contain CMakeLists.txt.
    Specify --help for usage, or press the help button on the CMake GUI.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-zrd6twth/libKMCUDA/setup.py", line 83, in <module>
        "Programming Language :: Python :: 3.6",
      File "/home/abhi/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/abhi/.local/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.6/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-build-zrd6twth/libKMCUDA/setup.py", line 19, in run
        self._build()
      File "/tmp/pip-build-zrd6twth/libKMCUDA/setup.py", line 37, in _build
        "."))
      File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '('cmake', '-DCMAKE_BUILD_TYPE=Release', '-DDISABLE_R=y', '-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda', '-DCUDA_ARCH=61', '.')' returned non-zero exit status 1.

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-zrd6twth/libKMCUDA/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-hiebv4_y-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-zrd6twth/libKMCUDA/

python 3.6.7 cuda 9.0

amnzaza commented 5 years ago

I have also tried that way and it seems that the CMakeLists.txt file is missing from the .tar itself. Could you please provide us with the file so that cmake can work correctly. Thank you.

vmarkovtsev commented 5 years ago

The workaround is to pip install git+https://github.com/src-d/kmcuda.git#subdirectory=src as written in the README.

Indeed, the current package seems broken - I don't see any files there. I am fixing now.

amnzaza commented 5 years ago

There is also an error for the cmake generator used. it needs to be of [arch] Win64 in order for CUDA library compilation. Is there a workaround to change the generator from my local computer? or do you as well need to change that yourself?

sajadtavakoli commented 5 years ago

hi Does this library also run on Windows?

smola commented 5 years ago

@sajadtavakoli No, it is currently not supported, see: https://github.com/src-d/kmcuda/issues/50

Please, avoid commenting on closed issues with unrelated questions. These will usually get unnoticed. Thank you!