I don't know if this is the correct way to express this. However, I was unable to run the project on the CUDA in the conda environment. After a lot of trial and error, I figured out the solution and wanted to share it here. I have tested this solution on different Python versions (namely 3.9, 3.10, and 3.12) and different CUDA versions (12.2.1 and 12.4).
I prepared my conda environment by following these steps:
Installing CUDA on conda environment. Use the command conda install nvidia::cuda to install CUDA on conda. Specific CUDA versions also could be installed. For example, use conda install nvidia/label/cuda-12.2.1::cuda to install version 12.2.1 documentation. in my case, CUDA 12.2.1 and 12.4 were both fine.
Set the environment variables. Use the following commands to set the environment variables within the conda environment. After executing these commands, you may need to reactivate the conda environment.
conda env config vars set CUDA_HOME="/home/user/miniconda3/envs/<ENV Name>/"
conda env config vars set LD_LIBRARY_PATH="/home/user/miniconda3/envs/<ENV Name>/targets/x86_64-linux/lib/:$LD_LIBRARY_PATH"
conda env config vars set CPATH = "/home/user/miniconda3/envs/<ENV Name>/targets/x86_64-linux/include/:$CPATH"
Clone and install the project using pip install -e .Note1: To find the correct path for CUDA_HOME use which nvcc. In my case, output of the command was:
>>> which nvcc
/home/user/miniconda3/envs/py12/bin/nvcc
Threefore, I set the CUDA_HOME as /home/user/miniconda3/envs/py12/.
Note2: To find the correct path for LD_LIBRARY_PATH use find ~ -name cuda_runtime_api.h. In my case, output of the command was:
So I set the LD_LIBRARY_PATH as /home/user/miniconda3/envs/py12/targets/x86_64-linux/lib/ and CPATH as /home/user/miniconda3/envs/py12/targets/x86_64-linux/include/. If you have multiple CUDA installations, the output of find ~ -name cuda_runtime_api.h will display multiple paths. Make sure to choose the path that corresponds to the environment you have created.
P.S. Here I include the errors I faced while finding this solution.
Error1: If I didn't set the CUDA_HOME correctly, I encountered the following issue. (Terminated log to save space)
running egg_info
creating groundingdino.egg-info
writing groundingdino.egg-info/PKG-INFO
writing dependency_links to groundingdino.egg-info/dependency_links.txt
writing requirements to groundingdino.egg-info/requires.txt
writing top-level names to groundingdino.egg-info/top_level.txt
writing manifest file 'groundingdino.egg-info/SOURCES.txt'
reading manifest file 'groundingdino.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'groundingdino.egg-info/SOURCES.txt'
running build_ext
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:414: UserWarning: The detected CUDA version (12.4) has a minor version mismatch with the version that was used to compile PyTorch (12.1). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 12.4
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
building 'groundingdino._C' extension
creating /home/user/workspace/groundingdino/GroundingDINO/build
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc
creating /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1965: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
warnings.warn(
Emitting ninja build file /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/4] /home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o
/home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
<command-line>: fatal error: cuda_runtime.h: No such file or directory
compilation terminated.
[2/4] /home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o
/home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
<command-line>: fatal error: cuda_runtime.h: No such file or directory
compilation terminated.
[3/4] c++ -MMD -MF /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o.d -pthread -B /home/user/miniconda3/envs/test/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o
c++ -MMD -MF /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o.d -pthread -B /home/user/miniconda3/envs/test/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
In file included from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContext.h:3,
from /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp:14:
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContextLight.h:6:10: fatal error: cuda_runtime_api.h: No such file or directory
6 | #include <cuda_runtime_api.h>
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
[4/4] c++ -MMD -MF /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/vision.o.d -pthread -B /home/user/miniconda3/envs/test/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/vision.cpp -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/vision.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
In file included from /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/vision.cpp:3:
/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn.h: In function ‘at::Tensor groundingdino::ms_deform_attn_forward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn.h:30:19: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
30 | if (value.type().is_cuda())
| ~~~~~~~~~~^~
In file included from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/core/Tensor.h:3,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/Tensor.h:3,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/function_hook.h:3,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/cpp_hook.h:2,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/variable.h:6,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/autograd/autograd.h:3,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/autograd.h:3,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/extension.h:5,
from /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.h:12,
from /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn.h:13:
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:225:30: note: declared here
225 | DeprecatedTypeProperties & type() const {
| ^~~~
/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn.h: In function ‘std::vector<at::Tensor> groundingdino::ms_deform_attn_backward(const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, const at::Tensor&, int)’:
/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn.h:52:19: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations]
52 | if (value.type().is_cuda())
| ~~~~~~~~~~^~
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:225:30: note: declared here
225 | DeprecatedTypeProperties & type() const {
| ^~~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2105, in _run_ninja_build
subprocess.run(
File "/home/user/miniconda3/envs/test/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/home/user/workspace/groundingdino/GroundingDINO/setup.py", line 204, in <module>
setup(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
return distutils.core.setup(**attrs)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 184, in setup
return run_commands(dist)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
dist.run_commands()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
self.run_command(cmd)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/develop.py", line 36, in run
self.install_for_development()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/develop.py", line 113, in install_for_development
self.run_command('build_ext')
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 93, in run
_build_ext.run(self)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
self.build_extensions()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 866, in build_extensions
build_ext.build_extensions(self)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
self._build_extensions_serial()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
self.build_extension(ext)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 254, in build_extension
_build_ext.build_extension(self, ext)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 560, in build_extension
objects = self.compiler.compile(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 679, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1785, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2121, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
Error2: If I didn't set the LD_LIBRARY_PATH correctly, I encountered the following issue. (Terminated log to save space)
running egg_info
writing groundingdino.egg-info/PKG-INFO
writing dependency_links to groundingdino.egg-info/dependency_links.txt
writing requirements to groundingdino.egg-info/requires.txt
writing top-level names to groundingdino.egg-info/top_level.txt
reading manifest file 'groundingdino.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'groundingdino.egg-info/SOURCES.txt'
running build_ext
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:414: UserWarning: The detected CUDA version (12.4) has a minor version mismatch with the version that was used to compile PyTorch (12.1). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 12.4
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
building 'groundingdino._C' extension
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1965: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
warnings.warn(
Emitting ninja build file /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/3] /home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o
/home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
<command-line>: fatal error: cuda_runtime.h: No such file or directory
compilation terminated.
[2/3] /home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o
/home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
<command-line>: fatal error: cuda_runtime.h: No such file or directory
compilation terminated.
[3/3] c++ -MMD -MF /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o.d -pthread -B /home/user/miniconda3/envs/test/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o
c++ -MMD -MF /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o.d -pthread -B /home/user/miniconda3/envs/test/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
In file included from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContext.h:3,
from /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp:14:
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContextLight.h:6:10: fatal error: cuda_runtime_api.h: No such file or directory
6 | #include <cuda_runtime_api.h>
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2105, in _run_ninja_build
subprocess.run(
File "/home/user/miniconda3/envs/test/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/home/user/workspace/groundingdino/GroundingDINO/setup.py", line 204, in <module>
setup(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
return distutils.core.setup(**attrs)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 184, in setup
return run_commands(dist)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
dist.run_commands()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
self.run_command(cmd)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/develop.py", line 36, in run
self.install_for_development()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/develop.py", line 113, in install_for_development
self.run_command('build_ext')
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 93, in run
_build_ext.run(self)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
self.build_extensions()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 866, in build_extensions
build_ext.build_extensions(self)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
self._build_extensions_serial()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
self.build_extension(ext)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 254, in build_extension
_build_ext.build_extension(self, ext)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 560, in build_extension
objects = self.compiler.compile(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 679, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1785, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2121, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
Error3: If I didn't set the CPATH correctly, I encountered the following issue. (Terminated log to save space)
running egg_info
writing groundingdino.egg-info/PKG-INFO
writing dependency_links to groundingdino.egg-info/dependency_links.txt
writing requirements to groundingdino.egg-info/requires.txt
writing top-level names to groundingdino.egg-info/top_level.txt
reading manifest file 'groundingdino.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'groundingdino.egg-info/SOURCES.txt'
running build_ext
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:414: UserWarning: The detected CUDA version (12.4) has a minor version mismatch with the version that was used to compile PyTorch (12.1). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 12.4
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
building 'groundingdino._C' extension
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1965: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
warnings.warn(
Emitting ninja build file /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/3] /home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o
/home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/cuda_version.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
<command-line>: fatal error: cuda_runtime.h: No such file or directory
compilation terminated.
[2/3] /home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o
/home/user/miniconda3/envs/test/bin/nvcc --generate-dependencies-with-compile --dependency-output /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o.d -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.cu -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_70,code=compute_70 -gencode=arch=compute_70,code=sm_70 -std=c++17
<command-line>: fatal error: cuda_runtime.h: No such file or directory
compilation terminated.
[3/3] c++ -MMD -MF /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o.d -pthread -B /home/user/miniconda3/envs/test/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
FAILED: /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o
c++ -MMD -MF /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o.d -pthread -B /home/user/miniconda3/envs/test/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -O2 -isystem /home/user/miniconda3/envs/test/include -fPIC -DWITH_CUDA -I/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/TH -I/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/THC -I/home/user/miniconda3/envs/test/include -I/home/user/miniconda3/envs/test/include/python3.10 -c -c /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp -o /home/user/workspace/groundingdino/GroundingDINO/build/temp.linux-x86_64-cpython-310/home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
In file included from /home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContext.h:3,
from /home/user/workspace/groundingdino/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn/ms_deform_attn_cpu.cpp:14:
/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContextLight.h:6:10: fatal error: cuda_runtime_api.h: No such file or directory
6 | #include <cuda_runtime_api.h>
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2105, in _run_ninja_build
subprocess.run(
File "/home/user/miniconda3/envs/test/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/home/user/workspace/groundingdino/GroundingDINO/setup.py", line 204, in <module>
setup(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
return distutils.core.setup(**attrs)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 184, in setup
return run_commands(dist)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
dist.run_commands()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
self.run_command(cmd)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/develop.py", line 36, in run
self.install_for_development()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/develop.py", line 113, in install_for_development
self.run_command('build_ext')
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
self.distribution.run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/dist.py", line 945, in run_command
super().run_command(command)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
cmd_obj.run()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 93, in run
_build_ext.run(self)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
self.build_extensions()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 866, in build_extensions
build_ext.build_extensions(self)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
self._build_extensions_serial()
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
self.build_extension(ext)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 254, in build_extension
_build_ext.build_extension(self, ext)
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 560, in build_extension
objects = self.compiler.compile(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 679, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1785, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/user/miniconda3/envs/test/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2121, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
I don't know if this is the correct way to express this. However, I was unable to run the project on the CUDA in the conda environment. After a lot of trial and error, I figured out the solution and wanted to share it here. I have tested this solution on different Python versions (namely 3.9, 3.10, and 3.12) and different CUDA versions (12.2.1 and 12.4).
I prepared my conda environment by following these steps:
conda install nvidia::cudato install CUDA on conda. Specific CUDA versions also could be installed. For example, useconda install nvidia/label/cuda-12.2.1::cudato install version 12.2.1 documentation. in my case, CUDA 12.2.1 and 12.4 were both fine.pip install -e .Note1: To find the correct path forCUDA_HOMEusewhich nvcc. In my case, output of the command was:Threefore, I set the
CUDA_HOMEas/home/user/miniconda3/envs/py12/.Note2: To find the correct path for
LD_LIBRARY_PATHusefind ~ -name cuda_runtime_api.h. In my case, output of the command was:So I set the
LD_LIBRARY_PATHas/home/user/miniconda3/envs/py12/targets/x86_64-linux/lib/andCPATHas/home/user/miniconda3/envs/py12/targets/x86_64-linux/include/. If you have multiple CUDA installations, the output offind ~ -name cuda_runtime_api.hwill display multiple paths. Make sure to choose the path that corresponds to the environment you have created.P.S. Here I include the errors I faced while finding this solution.
Error1: If I didn't set the
CUDA_HOMEcorrectly, I encountered the following issue. (Terminated log to save space)Error2: If I didn't set the
LD_LIBRARY_PATHcorrectly, I encountered the following issue. (Terminated log to save space)Error3: If I didn't set the
CPATHcorrectly, I encountered the following issue. (Terminated log to save space)