search

DIMESTeam / DIMES

MIT License
39 stars 6 forks source link

Installing torch_sampling failed. #1

Closed yangco-le closed 1 year ago

yangco-le commented 1 year ago

Dear authors, I kept the dependencies the same as you recommended, but I still failed to run pip install ./ torch_sampling.

Dependencies:

GPU: RTX 3090 CPU: AMD EPYC 7302 16-Core Processor

Command line output:

Processing ./torch_sampling
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: torch-sampling
  Building wheel for torch-sampling (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/liyang/miniconda3/envs/tsp/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/setup.py'"'"'; __file__='"'"'/mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-baockv2i
       cwd: /mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/
  Complete output (37 lines):
  Device: ['cpu', 'cuda']
  running bdist_wheel
  /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/utils/cpp_extension.py:339: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  running build
  running build_ext
  building 'torch_sampling' extension
  gcc -pthread -B /home/liyang/miniconda3/envs/tsp/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/TH -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.0/include -I/home/liyang/miniconda3/envs/tsp/include/python3.7m -c csrc/torch_scatter.cpp -o build/temp.linux-x86_64-cpython-37/csrc/torch_scatter.o -fPIC -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=torch_sampling -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  In file included from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/Parallel.h:149:0,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:12,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                   from csrc/torch_scatter.h:4,
                   from csrc/torch_scatter.cpp:8:
  /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
   #pragma omp parallel for if ((end - begin) >= grain_size)

  gcc -pthread -B /home/liyang/miniconda3/envs/tsp/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/TH -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.0/include -I/home/liyang/miniconda3/envs/tsp/include/python3.7m -c csrc/tsp.cpp -o build/temp.linux-x86_64-cpython-37/csrc/tsp.o -fPIC -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=torch_sampling -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  In file included from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/Parallel.h:149:0,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:12,
                   from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                   from csrc/tsp.cpp:3:
  /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
   #pragma omp parallel for if ((end - begin) >= grain_size)

  g++ -pthread -B /home/liyang/miniconda3/envs/tsp/compiler_compat -Wl,--sysroot=/ -pthread -shared -B /home/liyang/miniconda3/envs/tsp/compiler_compat -L/home/liyang/miniconda3/envs/tsp/lib -Wl,-rpath=/home/liyang/miniconda3/envs/tsp/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-cpython-37/csrc/torch_scatter.o build/temp.linux-x86_64-cpython-37/csrc/tsp.o -L/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/lib -L/usr/local/cuda-11.0/lib64 -l_segment_csr_cpu -l_segment_csr_cuda -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-37/torch_sampling.cpython-37m-x86_64-linux-gnu.so
  /home/liyang/miniconda3/envs/tsp/compiler_compat/ld: cannot find -l_segment_csr_cpu: No such file or directory
  /home/liyang/miniconda3/envs/tsp/compiler_compat/ld: cannot find -l_segment_csr_cuda: No such file or directory
  collect2: error: ld returned 1 exit status
  error: command '/usr/bin/g++' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for torch-sampling
  Running setup.py clean for torch-sampling
Failed to build torch-sampling
Installing collected packages: torch-sampling
    Running setup.py install for torch-sampling ... error
    ERROR: Command errored out with exit status 1:
     command: /home/liyang/miniconda3/envs/tsp/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/setup.py'"'"'; __file__='"'"'/mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-v7x2eyru/install-record.txt --single-version-externally-managed --compile --install-headers /home/liyang/miniconda3/envs/tsp/include/python3.7m/torch-sampling
         cwd: /mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/
    Complete output (43 lines):
    Device: ['cpu', 'cuda']
    running install
    /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      setuptools.SetuptoolsDeprecationWarning,
    running build
    running build_ext
    /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/utils/cpp_extension.py:339: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
      warnings.warn(msg.format('we could not find ninja.'))
    building 'torch_sampling' extension
    creating build
    creating build/temp.linux-x86_64-cpython-37
    creating build/temp.linux-x86_64-cpython-37/csrc
    gcc -pthread -B /home/liyang/miniconda3/envs/tsp/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/TH -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.0/include -I/home/liyang/miniconda3/envs/tsp/include/python3.7m -c csrc/torch_scatter.cpp -o build/temp.linux-x86_64-cpython-37/csrc/torch_scatter.o -fPIC -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=torch_sampling -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    In file included from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/Parallel.h:149:0,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:12,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                     from csrc/torch_scatter.h:4,
                     from csrc/torch_scatter.cpp:8:
    /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
     #pragma omp parallel for if ((end - begin) >= grain_size)

    gcc -pthread -B /home/liyang/miniconda3/envs/tsp/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/TH -I/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.0/include -I/home/liyang/miniconda3/envs/tsp/include/python3.7m -c csrc/tsp.cpp -o build/temp.linux-x86_64-cpython-37/csrc/tsp.o -fPIC -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=torch_sampling -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    In file included from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/Parallel.h:149:0,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/csrc/api/include/torch/all.h:12,
                     from /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/torch/extension.h:4,
                     from csrc/tsp.cpp:3:
    /home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/include/ATen/ParallelOpenMP.h:84:0: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
     #pragma omp parallel for if ((end - begin) >= grain_size)

    creating build/lib.linux-x86_64-cpython-37
    g++ -pthread -B /home/liyang/miniconda3/envs/tsp/compiler_compat -Wl,--sysroot=/ -pthread -shared -B /home/liyang/miniconda3/envs/tsp/compiler_compat -L/home/liyang/miniconda3/envs/tsp/lib -Wl,-rpath=/home/liyang/miniconda3/envs/tsp/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-cpython-37/csrc/torch_scatter.o build/temp.linux-x86_64-cpython-37/csrc/tsp.o -L/home/liyang/miniconda3/envs/tsp/lib/python3.7/site-packages/torch/lib -L/usr/local/cuda-11.0/lib64 -l_segment_csr_cpu -l_segment_csr_cuda -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-37/torch_sampling.cpython-37m-x86_64-linux-gnu.so
    /home/liyang/miniconda3/envs/tsp/compiler_compat/ld: cannot find -l_segment_csr_cpu: No such file or directory
    /home/liyang/miniconda3/envs/tsp/compiler_compat/ld: cannot find -l_segment_csr_cuda: No such file or directory
    collect2: error: ld returned 1 exit status
    error: command '/usr/bin/g++' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/liyang/miniconda3/envs/tsp/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/setup.py'"'"'; __file__='"'"'/mnt/nas2/home/liyang/DIMES/TSP/torch_sampling/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-v7x2eyru/install-record.txt --single-version-externally-managed --compile --install-headers /home/liyang/miniconda3/envs/tsp/include/python3.7m/torch-sampling Check the logs for full command output.
q-rz commented 1 year ago

Hi @yangco-le! Thank you for your interest in our work. The TSP code is now fully uploaded.

It seems that the installer of our torch_sampling failed in copying torch_scatter/_segment_csr_{cuda,cpu}.so from pip's package folder to lib_segment_csr_{cuda,cpu}.so in gcc's library folder. You may modify setup.py or copy them manually.