search

triton-inference-server / client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
BSD 3-Clause "New" or "Revised" License
551 stars 227 forks source link

fix is_contiguous_logic issue in dlpack #668

Open wjddn279 opened 4 months ago

wjddn279 commented 4 months ago

Hello, I'm making good use of the open source you provided. When I use it, I find an issue and fix it to contribute to your project

issue what I find

I want to use set_shared_memory_region_from_dlpack for GPU to GPU shared memory I converted pytorch tensor to dlpack using by to_dlpack from dlpack library but it raises error in this line

# it work well
img_tensor = img_tensor_raw.resize_(target_height, target_width, 3)
cudashm.set_shared_memory_region_from_dlpack(cuda_shm_ip_handle, [img_tensor])

# it doesn't work
img_tensor = img_tensor_raw.resize_(1, target_height, target_width, 3)
cudashm.set_shared_memory_region_from_dlpack(cuda_shm_ip_handle, [img_tensor])

although I make pytorch tensor contiguous, after making it dlpack strides corrupts it is known issue pytorch and the maintainer says it is not an issue but logic for preventing another issue. So to_dlpack make stride in tensor's dimension shape is 1 just 1

so, I fix is_contiguous logic to skip contiguous checking when shape is 1 and it work well in my code. If the contribution process is in the project, I will follow it. thanks

wjddn279 commented 4 months ago

The problem I've had is the same problem as this one https://github.com/triton-inference-server/client/issues/536 This PR can resolve the issue without lower version of pytorch below 1.12