NVIDIA / gdrcopy

A fast GPU memory copy library based on NVIDIA GPUDirect RDMA technology
MIT License
870 stars 143 forks source link

Install gdrcopy in docker images #197

Open TonyZion opened 3 years ago

TonyZion commented 3 years ago

Hello,
 We found gdrcopy can not be installed from source directly in docker images although root user, because command sudo ./insmod.sh can not be executed while using Dockerfile to build the image.
 We have to execute make prefix=<install-to-this-location> CUDA=<cuda-install-top-dir> all install only while using Dockerfile to build the image, and then execute docker run -it --privileged to create a container based on the image we built before executing sudo ./insmod.sh.
 Futhermore, we found the command sudo ./insmod.sh is not permanent, whcih means we must execute sudo ./insmod.sh everytime the container is created.
 Thus, my question is:

  1. Is the option --privileged required to install (or use) gdrcopy while creating a docker container?
  2. Please give advice to avoid executing sudo ./insmod.sh everytime the container is created.

 Thank you.

pakmarkthub commented 3 years ago

Hi,

gdrcopy composes of two parts: library and driver. For the driver, it is easier to install it on host. Then, you need to mount /dev/* into your container. If you do this, you just need to install only the library in your container. You can achieve that by sudo make lib_install inside your container.

To automatically load the driver, you would need a service. We install it as part of the gdrcopy driver package. You can make the package yourself using the script inside <gdrcopy-root>/packages/.