installation error

[Chen-Cai-OSU]

Hi, Thank you for the nice library. I am trying to install torchsort on a AWS machine with PyTorch 1.7.1 with Python3.7 (CUDA 11.1 and Intel MKL), and encounted the following error:

Looking in indexes: https://pypi.org/simple, https://pip.repos.neuron.amazonaws.com
Collecting torchsort
  Using cached torchsort-0.1.5.tar.gz (13 kB)
Requirement already satisfied: torch in /home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages (from torchsort) (1.8.1+cu111)
Requirement already satisfied: numpy in /home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages (from torch->torchsort) (1.19.2)
Requirement already satisfied: typing-extensions in /home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages (from torch->torchsort) (3.7.4.3)
Building wheels for collected packages: torchsort
  Building wheel for torchsort (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/setup.py'"'"'; __file__='"'"'/tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-sk1_6dbm
       cwd: /tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/
  Complete output (32 lines):
  running bdist_wheel
  /home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/utils/cpp_extension.py:369: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/torchsort
  copying torchsort/__init__.py -> build/lib.linux-x86_64-3.7/torchsort
  copying torchsort/ops.py -> build/lib.linux-x86_64-3.7/torchsort
  running egg_info
  writing torchsort.egg-info/PKG-INFO
  writing dependency_links to torchsort.egg-info/dependency_links.txt
  writing requirements to torchsort.egg-info/requires.txt
  writing top-level names to torchsort.egg-info/top_level.txt
  reading manifest file 'torchsort.egg-info/SOURCES.txt'
  writing manifest file 'torchsort.egg-info/SOURCES.txt'
  copying torchsort/isotonic_cpu.cpp -> build/lib.linux-x86_64-3.7/torchsort
  running build_ext
  building 'torchsort.isotonic_cpu' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/torchsort
  /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/x86_64-conda-linux-gnu-cc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include -fPIC -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/TH -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/THC -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/include/python3.7m -c torchsort/isotonic_cpu.cpp -o build/temp.linux-x86_64-3.7/torchsort/isotonic_cpu.o -fopenmp -ffast-math -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=isotonic_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
  /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/x86_64-conda-linux-gnu-c++ -pthread -shared -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -Wl,-rpath-link,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include build/temp.linux-x86_64-3.7/torchsort/isotonic_cpu.o -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-3.7/torchsort/isotonic_cpu.cpython-37m-x86_64-linux-gnu.so
  building 'torchsort.isotonic_cuda' extension
  /usr/local/cuda-11.1/bin/nvcc -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/TH -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.1/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/include/python3.7m -c torchsort/isotonic_cuda.cu -o build/temp.linux-x86_64-3.7/torchsort/isotonic_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=isotonic_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -ccbin /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/x86_64-conda-linux-gnu-cc -std=c++14
  x86_64-conda-linux-gnu-cc: error: torchsort/isotonic_cuda.cu: No such file or directory
  x86_64-conda-linux-gnu-cc: warning: '-x c++' after last input file has no effect
  x86_64-conda-linux-gnu-cc: fatal error: no input files
  compilation terminated.
  error: command '/usr/local/cuda-11.1/bin/nvcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for torchsort
  Running setup.py clean for torchsort
Failed to build torchsort
Installing collected packages: torchsort
    Running setup.py install for torchsort ... error
    ERROR: Command errored out with exit status 1:
     command: /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/setup.py'"'"'; __file__='"'"'/tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5t6z_gdw/install-record.txt --single-version-externally-managed --compile --install-headers /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include/python3.7m/torchsort
         cwd: /tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/
    Complete output (32 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/torchsort
    copying torchsort/__init__.py -> build/lib.linux-x86_64-3.7/torchsort
    copying torchsort/ops.py -> build/lib.linux-x86_64-3.7/torchsort
    running egg_info
    writing torchsort.egg-info/PKG-INFO
    writing dependency_links to torchsort.egg-info/dependency_links.txt
    writing requirements to torchsort.egg-info/requires.txt
    writing top-level names to torchsort.egg-info/top_level.txt
    /home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/utils/cpp_extension.py:369: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
      warnings.warn(msg.format('we could not find ninja.'))
    reading manifest file 'torchsort.egg-info/SOURCES.txt'
    writing manifest file 'torchsort.egg-info/SOURCES.txt'
    copying torchsort/isotonic_cpu.cpp -> build/lib.linux-x86_64-3.7/torchsort
    running build_ext
    building 'torchsort.isotonic_cpu' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/torchsort
    /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/x86_64-conda-linux-gnu-cc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include -fPIC -I/home/ubuntu/anaconda3/en
vs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/TH -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/THC -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/include/python3.7m -c torchsort/isotonic_cpu.cpp -o build/temp.linux-x86_64-3.7/torchsort/isotonic_cpu.o -fopenmp -ffast-math -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=isotonic_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
    cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
    /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/x86_64-conda-linux-gnu-c++ -pthread -shared -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -Wl,-rpath-link,/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include build/temp.linux-x86_64-3.7/torchsort/isotonic_cpu.o -L/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-3.7/torchsort/isotonic_cpu.cpython-37m-x86_64-linux-gnu.so
    building 'torchsort.isotonic_cuda' extension
    /usr/local/cuda-11.1/bin/nvcc -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/TH -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.1/include -I/home/ubuntu/anaconda3/envs/pytorch_latest_p37/include/python3.7m -c torchsort/isotonic_cuda.cu -o build/temp.linux-x86_64-3.7/torchsort/isotonic_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=isotonic_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -ccbin /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/x86_64-conda-linux-gnu-cc -std=c++14
    x86_64-conda-linux-gnu-cc: error: torchsort/isotonic_cuda.cu: No such file or directory
    x86_64-conda-linux-gnu-cc: warning: '-x c++' after last input file has no effect
    x86_64-conda-linux-gnu-cc: fatal error: no input files
    compilation terminated.
    error: command '/usr/local/cuda-11.1/bin/nvcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/ubuntu/anaconda3/envs/pytorch_latest_p37/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/setup.py'"'"'; __file__='"'"'/tmp/pip-install-55bpk6jp/torchsort_a99e8d8dd95647faafe6d8d72aa77601/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5t6z_gdw/install-record.txt --single-version-externally-managed --compile --install-headers /home/ubuntu/anaconda3/envs/pytorch_latest_p37/include/python3.7m/torchsort Check the logs for full command output.

Do do know what goes wrong？Let me know if u need any further info. Thanks!

[teddykoker]

Hi,

It looks like the relevant message is x86_64-conda-linux-gnu-cc: error: torchsort/isotonic_cuda.cu: No such file or directory. Which shouldn't be an issue since this file is in the source.

One thing you could try is installing manually:

git clone https://github.com/teddykoker/torchsort
cd torchsort
python setup.py install

Another issue I've seen with PyTorch C++ extensions is that using Ninja to build them has issues, in which case uninstalling Ninja would fall back to the default build method.

Let me know if these work!

[Chen-Cai-OSU]

install from source works! Thanks you for the help!

[teddykoker]

Looks like the file didn't actually get uploaded to PyPI for version 0.1.5. This has now been fixed in 0.1.6!