rusty1s / pytorch_scatter

PyTorch Extension Library of Optimized Scatter Operations
https://pytorch-scatter.readthedocs.io
MIT License
1.5k stars 178 forks source link

How to use pytorch_scatter functionality both from Python and C++? #406

Closed ArseniuML closed 6 months ago

ArseniuML commented 6 months ago

I have Pytorch 1.8.1 installed (this very version is needed). Next I have downloaded pytorch_scatter repo (version 2.1.0) from github and installed it like this:

pip install -e pytorch_scatter

After install succeeds, I can verify pytorch_scatter have been installed:

pip list | grep torch-scatter
torch-scatter                        2.1.0        /home/arseniy.marin@nami.local/Projects/FSDv2/pytorch_scatter

Now I want:

  1. Use pytorch_scatter functionality from Python (I already can use it)
  2. Use pytorch_scatter functinality from C++

I want to use existing C++ build from pytorch_scatter/build, which was created by "pip install", specifically scatter_cuda(..) function. This function seems to have been compiled in pytorch_scatter/build/lib.linux-x86_64-3.8/torch_scatter/_scatter_cuda.so. So I add the next lines to CMakeLists.txt:

set_target_properties(scatter_cuda PROPERTIES IMPORTED_LOCATION /home/arseniy.marin@nami.local/Projects/FSDv2/pytorch_scatter/build/lib.linux-x86_64-3.8/torch_scatter/_scatter_cuda.so)
target_link_libraries(test_cpp scatter_cuda)

test_cpp builds successfully, but fails in runtime:

error while loading shared libraries: libc10_cuda.so: cannot open shared object file: No such file or directory

I can locate libc10_cuda.so in Pytorch installation:

find . -name "libc10_cuda.so"
./lidarenv/lib/python3.8/site-packages/torch/lib/libc10_cuda.so

So how do I have to configure my C++ build properly? This is full CMakeLists.txt:

cmake_minimum_required(VERSION 3.0.0)
project(spconv_deploy VERSION 0.1.0 LANGUAGES C CXX)

include(CTest)
enable_testing()

set(CUDA_ARCHITECTURES "native")
set(CMAKE_CUDA_ARCHITECTURES ${CUDA_ARCHITECTURES})

find_package(Python3 COMPONENTS Development)
find_package(Torch REQUIRED PATHS ../../../../lidarenv/lib/python3.8/site-packages/torch/share/cmake)

include_directories(../../../../pytorch_scatter/csrc)
link_directories(../../../../pytorch_scatter/build/lib.linux-x86_64-3.8/torch-scatter)

add_library(scatter_cuda SHARED IMPORTED)
set_target_properties(scatter_cuda PROPERTIES
    IMPORTED_LOCATION /home/arseniy.marin@nami.local/Projects/FSDv2/pytorch_scatter/build/lib.linux-x86_64-3.8/torch_scatter/_scatter_cuda.so)

add_executable(test_cpp test_cpp.cpp)
target_link_libraries(test_cpp "${TORCH_LIBRARIES}" Python3::Python)
target_link_libraries(test_cpp scatter_cuda)

set(CPACK_PROJECT_NAME ${PROJECT_NAME})
set(CPACK_PROJECT_VERSION ${PROJECT_VERSION})
include(CPack)
ArseniuML commented 6 months ago

solved by adding

"cmake.environment": {
    "LD_LIBRARY_PATH": 
    "/home/arseniy.marin@nami.local/Projects/FSDv2/lidarenv/lib/python3.8/site-packages/torch/lib"
}

to launch.json of Visual Studio Code.