Open davew723 opened 2 years ago
I'm wondering if I'm doing something wrong here? Although the same approach is working with my mellanox cards. Example Issue:
Using AWS I setup two EC2 instances. Verify that security group allows me to ssh between the two instances. Them I install...
sudo yum groupinstall -y "Development Tools" sudo yum install -y cmake git sudo yum install -y rdma-core rdma-core-devel
... Next I setup my SoftRoCE interface.
sudo rdma link add rxe0 type rxe netdev eth0
Run ./rdma_server on the first EC2 instance ...
[ec2-user@ip-10-0-3-213 build]$ ./rdma_server rdma_server: start rdma_server: end 0
And rdma_client on the second...
[dev2:/home/ec2-user/git/rsockets/build]./rdma_client 10.0.3.213 rdma_client: start rdma_client: end 0
SUCCESS. Although rdma_client does NOT use https://github.com/linux-rdma/rdma-core/blob/master/librdmacm/rsocket.*
Next test rcopy in the same fashion.
[ec2-user@ip-10-0-3-213 build]$ ./rcopy waiting for connection...
[dev2:/home/ec2-user/git/rsockets/build]./rcopy /etc/hosts 10.0.3.213:/tmp/hosts rconnect failed : Invalid argument
FAIL Invalid argument. I tried a couple of the other rsockets examples like riostream and it fails similarly but on rpoll.
These same tests work fine with a mellanox adapter running RoCE.
Anyone else run into this?
I'm wondering if I'm doing something wrong here? Although the same approach is working with my mellanox cards. Example Issue:
Using AWS I setup two EC2 instances. Verify that security group allows me to ssh between the two instances. Them I install...
sudo yum groupinstall -y "Development Tools" sudo yum install -y cmake git sudo yum install -y rdma-core rdma-core-devel
... Next I setup my SoftRoCE interface.
sudo rdma link add rxe0 type rxe netdev eth0
Run ./rdma_server on the first EC2 instance ...
[ec2-user@ip-10-0-3-213 build]$ ./rdma_server rdma_server: start rdma_server: end 0
And rdma_client on the second...
[dev2:/home/ec2-user/git/rsockets/build]./rdma_client 10.0.3.213 rdma_client: start rdma_client: end 0
SUCCESS. Although rdma_client does NOT use https://github.com/linux-rdma/rdma-core/blob/master/librdmacm/rsocket.*
Next test rcopy in the same fashion.
[ec2-user@ip-10-0-3-213 build]$ ./rcopy waiting for connection...
[dev2:/home/ec2-user/git/rsockets/build]./rcopy /etc/hosts 10.0.3.213:/tmp/hosts rconnect failed : Invalid argument
FAIL Invalid argument. I tried a couple of the other rsockets examples like riostream and it fails similarly but on rpoll.
These same tests work fine with a mellanox adapter running RoCE.
Anyone else run into this?