search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
7.83k stars 1.43k forks source link

the parameters `triton_shm_name`, `shm_key` - why both are needed? #6447

Open BassamRamadan opened 9 months ago

BassamRamadan commented 9 months ago

the parameters triton_shm_name, shm_key - why both are needed in RegisterSystemSharedMemory? I assume shm_key is the name for shm_open. Why the other parameter? Why we not use the shm_key instead of triton_shm_name?

dyastremsky commented 9 months ago

I think triton_shm_name should be the readable name while shm_key might be the file location where that shared memory is stored.

CC: @Tabrizian, do you know the answer to this? We may want to improve the documentation, which should be sourced from here.

jbkyang-nvi commented 9 months ago

I believe we have this because of historical reasons as Cuda Shared Memory shares a same handle as System Shared Memory. shm_key should be able to be used for the SharedMemoryHandle as well. I think we should make a ticket for this.

dyastremsky commented 9 months ago

Thanks for clarifying, Katherine! Created a ticket for us to investigate that enhancement.

Ticket reference: DLIS-5719.