NVIDIA / cccl

CUDA Core Compute Libraries
https://nvidia.github.io/cccl/
Other
1.26k stars 163 forks source link

[BUG]: CUDA Thrust object shared between .cu and .cpp files #2737

Open RichardSmithLab opened 1 week ago

RichardSmithLab commented 1 week ago

Is this a duplicate?

Type of Bug

Compile-time Error

Component

Thrust

Describe the bug

It looks like recent changes to Thrust were made to tag the host_vector class name with the compile time CUDA architecture. This creates an issue (undefined reference) when linking shared objects compiled with nvcc or not (.cu vs .cpp files) that use Thrust objects, even host-only ones like host_vector.

Compiling everything with nvcc is probably in our case is not an option, some of the files need moc (ie Qt).

How to Reproduce

To reproduce on Ubuntu 20.04.6 LTS with Cuda 12.6: Main.cpp:

include include <thrust/host_vector.h>

bool cudaFunc(thrust::host_vector &vec); int main() { thrust::host_vector vec; cudaFunc(vec); }

cudaFunc.cu:

include include <thrust/host_vector.h> bool cudaFunc(thrust::host_vector &vec) { return true; }

Compile with:

/usr/local/cuda/bin/nvcc -c cudaFunc.cu g++ -I /usr/local/cuda/include -L /usr/local/cuda/targets/x86_64-linux/lib main.cpp cudaFunc.o -lcudart -ldl

will produce the error:

/usr/bin/ld: /tmp/cc5fFAJC.o: in function main: main.cpp:(.text+0x30): undefined reference to cudaFunc(thrust::THRUST_200500_CUDA_ARCHLISTNS::host_vector<int, std::allocator >&)

This will link fine on Ubuntu 20 with Cuda 12.3.

Expected behavior

No link error.

Reproduction link

No response

Operating System

No response

nvidia-smi output

No response

NVCC version

nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2024 NVIDIA Corporation Built on Thu_Sep_12_02:18:05_PDT_2024 Cuda compilation tools, release 12.6, V12.6.77 Build cuda_12.6.r12.6/compiler.34841621_0

jrhemstad commented 1 week ago

Unfortunately, this is a known limitation that resulted from the solution we used to solve an even more insidious problem. See: https://github.com/NVIDIA/cccl?tab=readme-ov-file#application-binary-interface-abi

The good news is that for most Thrust types, we have a replacement from cuda:: that does work between .cu and .cpp files. thrust::host_vector is one that we don't have an immediate replacement for, but that work is actively ongoing.

RichardSmithLab commented 1 week ago

Thank you for your reply. In my .o file compiled with nvcc, host_vector looks like: thrust::THRUST_200500_520_NS::host_vector

Can you see an easy way to get thrust headers to tag the name correctly when compiling with g++? (ie swap " _CUDA_ARCHLIST" for "520"? I would be OK with making small changes to the Thrust headers to get this to compile until a cuda::host_vector or similar comes along, which I guess will be quite some time in the future. I was thinking something like redefining THRUST_DETAIL_ABI_NS_BEGIN/END in the case when THRUST_DEVICE_SYSTEM != THRUST_DEVICE_SYSTEM_CUDA.