Hi,
test system is IBM AC922, 2 x TESLA V100, Connectx-5EN back to back with a x86_64 DELL R740
using RoCEv2 (UC queue pair, WRITE verbs), I have 97Gb/s BW to AC922 CPU memory but only 39 Gb/s to TESLA memory
nv_peer_mem and nv_rsync_mem module are loaded, nvidia-persistenced started
MLNX OFED installed
same issue with either custom code or perftest/ib_write_bw.
observed BW between 2xDELL GPU Quadro P6000 as expected
Hi, test system is IBM AC922, 2 x TESLA V100, Connectx-5EN back to back with a x86_64 DELL R740
using RoCEv2 (UC queue pair, WRITE verbs), I have 97Gb/s BW to AC922 CPU memory but only 39 Gb/s to TESLA memory nv_peer_mem and nv_rsync_mem module are loaded, nvidia-persistenced started MLNX OFED installed
same issue with either custom code or perftest/ib_write_bw.
observed BW between 2xDELL GPU Quadro P6000 as expected