search

Mellanox / libvma

Linux user space library for network socket acceleration based on RDMA compatible network adaptors
https://www.mellanox.com/products/software/accelerator-software/vma?mtag=vma
Other
585 stars 153 forks source link

How to work with the missing LD_LIBRARY_PATH after setcap? #1091

Open kc-eos opened 2 months ago

kc-eos commented 2 months ago

Hello:

I am trying to run my project with libvma as non-root user, it is fine to run the sample sockperf with libvma after setting up the necessary permission with setuid and setcap mentioned in the user guide.

However, when I try to run it with my project, I found that LD_LIBRARY_PATH is missing / ignored by LD linker as it is running in a "secure-execution" mode after granting the capabilities to the executable. My project is in a structure where the executable has to link with several inhouse-built library, e.g.

 my_gateway -> libmy-core.so , libmy-network.so, libmy-util.so
 my_engine -> libmy-core.so , libmy-network.so, libmy-util.so
...

Then, without the LD_LIBRARY_PATH, I cannot start up my application due to ... : error while loading shared libraries: ....

I wonder

  1. is there any recommendation to workaround this security-protection; and
  2. will it be possible to run libvma with a wrapper script such that I don't need to run setcap on all the executables

PS1. Tried to bind rpath during compilation, but it is not perfect as the same .so will be deployed to different account without a consistent path. PS2. It is running on RHEL 7.4

igor-ivanov commented 2 weeks ago

Hello, try to load libvma using dlopen. See https://docs.nvidia.com/networking/display/vmav972lts/vma+configuration#src-106008374_VMAConfiguration-LoadingVMADynamically

kc-eos commented 2 weeks ago

Hi @igor-ivanov , thanks for the reply.

I want to confirm my understanding.

If we do dlopen, does libvma still need certain Linux capabilities to bypass the network stack? If so, how can we startup the process without doing setcap on the executable (which cause LD_LIBRARY_PATH not effective).

I have tried to setup the capabilities programmatically (via libcap), but it's also not working because, by design, Linux capabilities can only be dropped, but never be raised at runtime.

igor-ivanov commented 2 weeks ago

Related permissions are still needed in dlopen case too.