Open dusty-nv opened 1 year ago
Looks like indeed we didn't have proper kDLCUDAManaged
. I think it can be added in similar ways as https://github.com/apache/tvm/blob/main/src/runtime/cuda/cuda_device_api.cc#L256
And add allocation support. I don't have experience with CUDAManaged
before, but if it is a matter of updating cudamalloc and copy we might be able to update support
Hello, I am using an NVIDIA Jetson device which has unified CPU/GPU memory, and I'm trying to eliminate unneeded CPU<->GPU memory copies. I noticed there are
kDLCUDAManaged
andkDLCUDAHost
device types. But when I try to construct a device withkDLCUDAManaged
:What is the proper way to use
kDLCUDAManaged
?Environment
TVM Unity 0.12 Ubuntu 20.04, JetPack 5.1.2, CUDA 11.4