Hi,I meet some questions when I verify that GDS installation was successfulI. I've already installed MLNX_OFED driver and had one NVMe SSD. But when I run ./gdscheck.py -p, it's still showing that "NVMe : Unsupported" and "--Mellanox PeerDirect : Disabled". I want to know why and how to solve them. Thanks. The results are as follows.
（cuda:11.7 , nvidia driver:515.43.04 , MLNX_OFED：5.6，uname -r (kernel): 5.4.0 , SSD: optane NVMe SSD , GPU:P100 ）
Pedrexus
commented
1 year ago
Hi,I meet some questions when I verify that GDS installation was successfulI. I've already installed MLNX_OFED driver and had one NVMe SSD. But when I run ./gdscheck.py -p, it's still showing that "NVMe : Unsupported" and "--Mellanox PeerDirect : Disabled". I want to know why and how to solve them. Thanks. The results are as follows. （cuda:11.7 , nvidia driver:515.43.04 , MLNX_OFED：5.6，uname -r (kernel): 5.4.0 , SSD: optane NVMe SSD , GPU:P100 ）
============ ENVIRONMENT:
DRIVER CONFIGURATION: NVMe : Unsupported NVMeOF : Unsupported SCSI : Unsupported ScaleFlux CSD : Unsupported NVMesh : Unsupported DDN EXAScaler : Unsupported IBM Spectrum Scale : Unsupported NFS : Unsupported BeeGFS : Unsupported WekaFS : Unsupported Userspace RDMA : Unsupported --Mellanox PeerDirect : Disabled --rdma library : Not Loaded (libcufile_rdma.so) --rdma devices : Not configured --rdma_device_status : Up: 0 Down: 0 CUFILE CONFIGURATION: properties.use_compat_mode : true properties.force_compat_mode : false properties.gds_rdma_write_support : true properties.use_poll_mode : false properties.poll_mode_max_size_kb : 4 properties.max_batch_io_size : 128 properties.max_batch_io_timeout_msecs : 5 properties.max_direct_io_size_kb : 16384 properties.max_device_cache_size_kb : 131072 properties.max_device_pinned_mem_size_kb : 33554432 properties.posix_pool_slab_size_kb : 4 1024 16384 properties.posix_pool_slab_count : 128 64 32 properties.rdma_peer_affinity_policy : RoundRobin properties.rdma_dynamic_routing : 0 fs.generic.posix_unaligned_writes : false fs.lustre.posix_gds_min_kb: 0 fs.beegfs.posix_gds_min_kb: 0 fs.weka.rdma_write_support: false profile.nvtx : false profile.cufile_stats : 0 miscellaneous.api_check_aggressive : false GPU INFO: GPU index 0 Tesla P100-PCIE-16GB bar:1 bar size (MiB):16384 supports GDS PLATFORM INFO: IOMMU: disabled Platform verification succeeded