search

teddykoker / torchsort

Fast, differentiable sorting and ranking in PyTorch
https://pypi.org/project/torchsort/
Apache License 2.0
765 stars 33 forks source link

Unable to install either via pip or from source in docker #40

Closed yilmazff closed 2 years ago

yilmazff commented 2 years ago

I want to install in a docker container created using NVIDIA Container Toolkit.

However, I am getting the following error message every time despite trying out all the alternative routes that are listed in the README or in other issues:

root:/workspace# pip install torchsort
WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting torchsort
  Downloading torchsort-0.1.7.tar.gz (11 kB)
Requirement already satisfied: torch in /opt/conda/lib/python3.8/site-packages (from torchsort) (1.8.1)
Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.8/site-packages (from torch->torchsort) (3.7.4.3)
Requirement already satisfied: numpy in /root/.local/lib/python3.8/site-packages (from torch->torchsort) (1.21.0)
Building wheels for collected packages: torchsort
  Building wheel for torchsort (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-s3448tu7/torchsort/setup.py'"'"'; __file__='"'"'/tmp/pip-install-s3448tu7/torchsort/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-kmuaqe1f
       cwd: /tmp/pip-install-s3448tu7/torchsort/
  Complete output (60 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/torchsort
  copying torchsort/__init__.py -> build/lib.linux-x86_64-3.8/torchsort
  copying torchsort/ops.py -> build/lib.linux-x86_64-3.8/torchsort
  running egg_info
  writing torchsort.egg-info/PKG-INFO
  writing dependency_links to torchsort.egg-info/dependency_links.txt
  writing requirements to torchsort.egg-info/requires.txt
  writing top-level names to torchsort.egg-info/top_level.txt
  reading manifest file 'torchsort.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'torchsort.egg-info/SOURCES.txt'
  copying torchsort/isotonic_cpu.cpp -> build/lib.linux-x86_64-3.8/torchsort
  copying torchsort/isotonic_cuda.cu -> build/lib.linux-x86_64-3.8/torchsort
  running build_ext
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-s3448tu7/torchsort/setup.py", line 52, in <module>
      setup(
    File "/opt/conda/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/opt/conda/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/opt/conda/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 290, in run
      self.run_command('build')
    File "/opt/conda/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.8/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/opt/conda/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/opt/conda/lib/python3.8/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 378, in build_extensions
      self._check_abi()
    File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 734, in _check_abi
      check_compiler_abi_compatibility(compiler)
    File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 282, in check_compiler_abi_compatibility
      if not check_compiler_ok_for_platform(compiler):
    File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 242, in check_compiler_ok_for_platform
      which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT)
    File "/opt/conda/lib/python3.8/subprocess.py", line 415, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/opt/conda/lib/python3.8/subprocess.py", line 516, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['which', 'g++']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for torchsort
  Running setup.py clean for torchsort
Failed to build torchsort
Installing collected packages: torchsort
    Running setup.py install for torchsort ... error
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-s3448tu7/torchsort/setup.py'"'"'; __file__='"'"'/tmp/pip-install-s3448tu7/torchsort/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-r_bd2xc3/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.8/torchsort
         cwd: /tmp/pip-install-s3448tu7/torchsort/
    Complete output (62 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/torchsort
    copying torchsort/__init__.py -> build/lib.linux-x86_64-3.8/torchsort
    copying torchsort/ops.py -> build/lib.linux-x86_64-3.8/torchsort
    running egg_info
    writing torchsort.egg-info/PKG-INFO
    writing dependency_links to torchsort.egg-info/dependency_links.txt
    writing requirements to torchsort.egg-info/requires.txt
    writing top-level names to torchsort.egg-info/top_level.txt
    reading manifest file 'torchsort.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'torchsort.egg-info/SOURCES.txt'
    copying torchsort/isotonic_cpu.cpp -> build/lib.linux-x86_64-3.8/torchsort
    copying torchsort/isotonic_cuda.cu -> build/lib.linux-x86_64-3.8/torchsort
    running build_ext
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-s3448tu7/torchsort/setup.py", line 52, in <module>
        setup(
      File "/opt/conda/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/opt/conda/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/opt/conda/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/opt/conda/lib/python3.8/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/opt/conda/lib/python3.8/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/opt/conda/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/opt/conda/lib/python3.8/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/opt/conda/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/opt/conda/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
        _build_ext.run(self)
      File "/opt/conda/lib/python3.8/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 378, in build_extensions
        self._check_abi()
      File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 734, in _check_abi
        check_compiler_abi_compatibility(compiler)
      File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 282, in check_compiler_abi_compatibility
        if not check_compiler_ok_for_platform(compiler):
      File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 242, in check_compiler_ok_for_platform
        which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT)
      File "/opt/conda/lib/python3.8/subprocess.py", line 415, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      File "/opt/conda/lib/python3.8/subprocess.py", line 516, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['which', 'g++']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-s3448tu7/torchsort/setup.py'"'"'; __file__='"'"'/tmp/pip-install-s3448tu7/torchsort/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-r_bd2xc3/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.8/torchsort Check the logs for full command output.

 

When I try to install from the source by cloning and running python setup.py install, I get the following error that appears to be identical to the issue encountered via pip:

root:/workspace/torchsort# python setup.py install
running install
running bdist_egg
running egg_info
creating torchsort.egg-info
writing torchsort.egg-info/PKG-INFO
writing dependency_links to torchsort.egg-info/dependency_links.txt
writing requirements to torchsort.egg-info/requires.txt
writing top-level names to torchsort.egg-info/top_level.txt
writing manifest file 'torchsort.egg-info/SOURCES.txt'
reading manifest file 'torchsort.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'torchsort.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/torchsort
copying torchsort/__init__.py -> build/lib.linux-x86_64-3.8/torchsort
copying torchsort/ops.py -> build/lib.linux-x86_64-3.8/torchsort
copying torchsort/isotonic_cpu.cpp -> build/lib.linux-x86_64-3.8/torchsort
copying torchsort/isotonic_cuda.cu -> build/lib.linux-x86_64-3.8/torchsort
running build_ext
Traceback (most recent call last):
  File "setup.py", line 52, in <module>
    setup(
  File "/opt/conda/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/opt/conda/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/conda/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.8/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/opt/conda/lib/python3.8/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/opt/conda/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 167, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/opt/conda/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 153, in call_command
    self.run_command(cmdname)
  File "/opt/conda/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/opt/conda/lib/python3.8/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/opt/conda/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/opt/conda/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 378, in build_extensions
    self._check_abi()
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 734, in _check_abi
    check_compiler_abi_compatibility(compiler)
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 282, in check_compiler_abi_compatibility
    if not check_compiler_ok_for_platform(compiler):
  File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 242, in check_compiler_ok_for_platform
    which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT)
  File "/opt/conda/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/opt/conda/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['which', 'g++']' returned non-zero exit status 1.

 

Again, note that I've gone through the following steps in the README before running any of the above:

Conda Installation On some systems the package my not compile with pip install in conda environments. If this happens you may need to: Install g++ with conda install -c conda-forge gxx_linux-64 Set export variable export CXX=/path/to/miniconda3/envs/env_name/bin/x86_64-conda_cos6-linux-gnu-g++ If still failing, export variable export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/miniconda3/lib Thanks to @levnikmyskin for pointing this out!

teddykoker commented 2 years ago

It looks like g++ may not be installed in the image. Maybe try adding sudo apt install g++ to the dockerfile? I am currently away from my CUDA system, but I will try to reproduce this shortly.

yilmazff commented 2 years ago

Since I did conda install -c conda-forge gxx_linux-64, I never thought of properly installing g++ with apt install g++.

Thanks a lot for the help, the issue is resolved with that.