Use boost::interprocess::shared_memory_object i.s.o POSIX shared memory functions

[PJKuyten]

Is your feature request related to a problem? Please describe. Currently there is no way to use shared memory between triton-server and client apps on Windows . This is caused by the fact that the shared memory implementation is built upon the POSIX API for shared memory. On Windows builds all shared memory functions stub into error returning variants. Only Linux and other OSses that have a POSIX API implementation are supported for the use of shared memory.

Describe the solution you'd like In the boost::interprocess library there is a universal solution for shared memory that works for all OSses. It is called shared_memory_object. This works by using the POSIX API on Linux and simulates this API by using memory mapped files on Windows. Together with the mapped_region class (which allows the mapping of shared memory into a processes address space) they can fulfill all the functions that are needed for a shared memory implementation that is usable on both Windows and Linux OS.

So please implement this change so that Windows users of triton-server can also use shared memory with triton-server executables and client applications.

Describe alternatives you've considered The only alternative to shared memory usage, to get good performance, is the Server API, which requires the linking of triton-server as a library into the client application. The Server API is however quite low level and would make the Windows client application quite different from the Linux application (which uses the gRPC API). Having to maintain two completely different client applications is not desirable.

Additional context Boosts documentation for shared memory => https://www.boost.org/doc/libs/1_81_0/doc/html/interprocess/sharedmemorybetweenprocesses.html

[GuanLuo]

Thanks for the feature request, marking as enhancement

[maorb112]

Hello Nvidia,

There is any update about that?

Thanks, Maor