Unable to install nv_peer_mem on CentOS 8.2 /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/nv_peer_mem.c:46:10: fatal error: rdma/peer_mem.h: No such file or directory

[hanyunfan]

[root@node008 nv_peer_memory]# rpmbuild --rebuild /tmp/nvidia_peer_memory-1.1-0.src.rpm Installing /tmp/nvidia_peer_memory-1.1-0.src.rpm Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.iaPxvC

• umask 022
• cd /usr/src/redhat/BUILD
• cd /usr/src/redhat/BUILD
• rm -rf nvidia_peer_memory-1.1
• /usr/bin/gzip -dc /usr/src/redhat/SOURCES/nvidia_peer_memory-1.1.tar.gz
• /usr/bin/tar -xvvof - drwxr-xr-x root/root 0 2021-03-17 09:14 nvidia_peer_memory-1.1/ -rw-r--r-- root/root 4949 2021-03-17 09:14 nvidia_peer_memory-1.1/Makefile -rw-r--r-- root/root 3423 2021-03-17 09:14 nvidia_peer_memory-1.1/README.md -rwxr-xr-x root/root 2281 2021-03-17 09:14 nvidia_peer_memory-1.1/build_module.sh -rw-r--r-- root/root 5817 2021-03-17 09:14 nvidia_peer_memory-1.1/compat_nv-p2p.h -rwxr-xr-x root/root 4211 2021-03-17 09:14 nvidia_peer_memory-1.1/create_nv.symvers.sh -rw-r--r-- root/root 614 2021-03-17 09:14 nvidia_peer_memory-1.1/dkms.conf -rwxr-xr-x root/root 2756 2021-03-17 09:14 nvidia_peer_memory-1.1/nv_peer_mem -rwxr-xr-x root/root 13013 2021-03-17 09:14 nvidia_peer_memory-1.1/nv_peer_mem.c -rw-r--r-- root/root 47 2021-03-17 09:14 nvidia_peer_memory-1.1/nv_peer_mem.conf -rwxr-xr-x root/root 241 2021-03-17 09:14 nvidia_peer_memory-1.1/nv_peer_mem.upstart -rw-r--r-- root/root 3299 2021-03-17 09:14 nvidia_peer_memory-1.1/nvidia_peer_memory.spec drwxr-xr-x root/root 0 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/ -rw-r--r-- root/root 2366 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/changelog -rw-r--r-- root/root 2 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/compat -rw-r--r-- root/root 910 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/control -rw-r--r-- root/root 10 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/nvidia-peer-memory-dkms.dkms -rw-r--r-- root/root 24 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/nvidia-peer-memory-dkms.install -rw-r--r-- root/root 245 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/nvidia-peer-memory-dkms.postinst -rw-r--r-- root/root 81 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/nvidia-peer-memory.install -rwxr-xr-x root/root 198 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/nvidia-peer-memory.postinst -rwxr-xr-x root/root 199 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/nvidia-peer-memory.prerm -rwxr-xr-x root/root 557 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/rules -rwxr-xr-x root/root 431 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/updateInit.sh drwxr-xr-x root/root 0 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/patches/ -rw-r--r-- root/root 369 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/patches/dkms_name.patch -rw-r--r-- root/root 16 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/patches/series drwxr-xr-x root/root 0 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/source/ -rw-r--r-- root/root 13 2021-03-17 09:14 nvidia_peer_memory-1.1/debian/source/format
• STATUS=0
• '[' 0 -ne 0 ']'
• cd nvidia_peer_memory-1.1
• /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
• exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.Hl8cnN
• umask 022
• cd /usr/src/redhat/BUILD
• cd nvidia_peer_memory-1.1
• export KVER=4.18.0-193.el8.x86_64
• KVER=4.18.0-193.el8.x86_64
• make KVER=4.18.0-193.el8.x86_64 all INFO: Building with Inbox InfiniBand Stack Makefile:15: "WARNING: Compilation might fail against Inbox InfiniBand Stack as it might lack needed support; in such cases you need to install MLNX_OFED package first." /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/create_nv.symvers.sh 4.18.0-193.el8.x86_64 Getting symbol versions from /lib/modules/4.18.0-193.el8.x86_64/kernel/drivers/video/nvidia.ko ... Created: /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/nv.symvers Found /usr/src/nvidia-460.32.03//nvidia/nv-p2p.h /bin/cp -f /usr/src/nvidia-460.32.03//nvidia/nv-p2p.h /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/nv-p2p.h echo -n "" > my.symvers cat nv.symvers >> my.symvers make -C /lib/modules/4.18.0-193.el8.x86_64/build M=/usr/src/redhat/BUILD/nvidia_peer_memory-1.1 KBUILD_EXTRA_SYMBOLS="/usr/src/redhat/BUILD/nvidia_peer_memory-1.1/my.symvers" modules make[1]: Entering directory '/usr/src/kernels/4.18.0-193.el8.x86_64' INFO: Building with Inbox InfiniBand Stack /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/Makefile:15: "WARNING: Compilation might fail against Inbox InfiniBand Stack as it might lack needed support; in such cases you need to install MLNX_OFED package first." awk: fatal: cannot open file `nvidia_peer_memory.spec' for reading (No such file or directory) CC [M] /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/nv_peer_mem.o /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/nv_peer_mem.c:46:10: fatal error: rdma/peer_mem.h: No such file or directory

  include <rdma/peer_mem.h>

    ^~~~~~~~~~~~~~~~~

  compilation terminated. make[2]: [scripts/Makefile.build:319: /usr/src/redhat/BUILD/nvidia_peer_memory-1.1/nv_peer_mem.o] Error 1 make[1]: [Makefile:1542: module/usr/src/redhat/BUILD/nvidia_peer_memory-1.1] Error 2 make[1]: Leaving directory '/usr/src/kernels/4.18.0-193.el8.x86_64' make: *** [Makefile:73: all] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.Hl8cnN (%build)

RPM build errors: Bad exit status from /var/tmp/rpm-tmp.Hl8cnN (%build) [root@node008 nv_peer_memory]# uname -a Linux node008 4.18.0-193.el8.x86_64 #1 SMP Fri May 8 10:59:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux [root@node008 nv_peer_memory]#

[loveshack]

If you get peer_mem.h from MOFED and copy it into the Linux' include/rdma, you can build the module. Unfortunately the result needs MOFED to run. When you load it, you'll see it needs symbols from ib_core which aren't in the kernel.org RDMA support (or added to RHEL, SUSE, or Debian).