search

SoftRoCE / rxe-dev

Development Repository for RXE
Other
128 stars 55 forks source link

IB test fails for ibv_rc_pingpong and ibv_read_bw under RHEL6 #47

Open cywong opened 8 years ago

cywong commented 8 years ago

I followed instruction from https://github.com/SoftRoCE/rxe-dev/wiki/rxe-dev:-Home to configure and install SoftROCE, rping and ibv_write_bw works but no response for ibv_rc_pingpong and ibv_read_bw.

How to troubleshoot this issue?

Environment Two VMs of CentOS 6.7 (Using rxe 4.0.0+ kernel) SoftROCE: rxe-dev: ec91e948b67c2075cfe2d9b255b74d7e5a225064 librxe-dev: fd829a95921cfc3f8e1108fcf84d7e8ac64f4bca

_rxecfg status

Name  Link  Driver  Speed  NMTU  IPv4_addr        RDEV  RMTU
eth0  yes   e1000          1500  192.168.122.219  rxe0  1024  (3)

_ibv_rcpingpong

[root@test]# ibv_rc_pingpong -d rxe0 -g 0 <Peer IP>
  local address:  LID 0x0000, QPN 0x000011, PSN 0x017d96, GID fe80::5054:ff:fe6c:16f1
  remote address: LID 0x0000, QPN 0x000011, PSN 0xd7af63, GID fe80::5054:ff:fead:b03
**--Hang--**

_ibv_readbw

Device not recognized to implement inline feature. Disabling it
.....
---------------------------------------------------------------------------------------
                    RDMA_Read BW Test
 Dual-port       : OFF      Device         : rxe0
 Number of qps   : 1        Transport type : IB
 Connection type : RC       Using SRQ      : OFF
 CQ Moderation   : 100
 Mtu             : 1024[B]
 Link type       : Ethernet
 Gid index       : 0
 Outstand reads  : 128
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0011 PSN 0xcb231f OUT 0x80 RKey 0x000411 VAddr 0x007f696bb5e000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:30
 remote address: LID 0000 QPN 0x0011 PSN 0x71e12d OUT 0x80 RKey 0x000411 VAddr 0x007f83f17ff000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:192:168:122:219
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
**--Hang--**

/var/log/messages test kernel: no qp matches qpn 0x11

psebexen commented 8 years ago

I notice in your pasted input below ibv_rc_pingpong you include the argument '-g 0'. If you have not already, please try running the tests with '-g 1' on both machines. You should be able list all of the GID entries with cat /sys/class/infiniband/rxe0/ports/1/gids/*. The first entry (number 0) is the link local GID only; entry 1 should be used for testing purposes.

The ibv_read_bw test may benefit from adding the '-R' flag. This will attempt to connect using rdma_cm instead of the default Ethernet approach.